Testing the Tests: Using CPM to Reveal the Systems Cognitive Tests are Reflecting
Poster No:
1409
Submission Type:
Abstract Submission
Authors:
Anja Samardzija1, Xilin Shen2, Wenjing Luo2, Abigail Greene2, Saloni Mehta1, Fuyuze Tokoglu2, Jagriti Arora1, Scott Woods2, Rachel Katz2, Gerard Sanacora2, Vinod Srihari2, Dustin Scheinost2, Todd Constable1
Institutions:
1Yale University, New Haven, CT, 2Yale School of Medicine, New Haven, CT
First Author:
Co-Author(s):
Introduction:
Most connectome-based predictive modeling (CPM) can be conceptualized as feed-forward, as it uses some external phenotypic measure to reveal the brain circuits supporting test score. Here, we reformulate CPM by predefining networks for each cognitive construct and quantifying how much each brain network varies with test performance for 16 test scores frequently used in cognitive phenotyping. Predictive performance for each network, in this feedback sense, reflects the degree to which a particular test relies on specific brain networks. The goal is not to identify the circuit supporting an external measure but to assume we have networks of interest and to ask what each external measure reflects. By providing an interpretable quantitative measure, this work introduces a framework for developing improved phenotypic measures that interrogate specific brain networks.
Methods:
A transdiagnostic fMRI dataset – consisting of two rest and six task runs - was collected at Yale from 227 demographically diverse subjects, including healthy participants and patients. The Shen268 (Shen, 2013) atlas was applied to the data. The mean time courses of each node pair were correlated, generating eight (for the eight runs) 268×268 connectivity matrices per subject from which the construct networks were extracted. While any network of interest can be predefined, here, we defined the six cognitive construct networks: attention, perception, declarative memory, language, cognitive control, and working memory. The networks are defined as the edges between the nodes extracted from NeuroSynth (https://neurosynth.org): the construct term was input, and coordinates from the composite maps were extracted and associated with Shen268 nodes. In the connectivity matrices, the entire row for each node selected for a given construct was included in its network. The cognitive construct networks for each imaging run vary from 7×268 to 12×268 nodes. Predictive modeling of each construct network on each test measure was performed using separate kernel ridge regression models. The connectivity matrices from the two rest and the six task runs were concatenated. Thus, the input feature to the predictive model consists of edges of matrices that vary in size from 7×268×8 to 12×268×8. Predictive power is defined as the correlation between the predicted and the observed behavioral measure.
Results:
Fig. 1 shows the predictive power of the construct networks on 16 tests (mean predictive power ~0.21). For each network, we rank the tests according to the predictive power for that network. Higher predictive power for a given construct network implies a larger role for that network in task performance. The attention network, e.g., plays a major role in Boston naming test (Kaplan, 2001) performance but a minimal role on the cancellation task (Lichtenberger, 2012), while the language network has the highest contribution on matrix reasoning (Wechsler, 1999) and lowest on 20 questions (Delis, 2001). For a given task, the predictive performance of each construct network within a test is observed in Fig. 2. Here, the predictive power reflects the contribution of each network to performance on a given test. E.g., for color-word interference (left column), a subtest of D-KEFS (Delis, 2001) that improves on the Stroop test (Golden, 1978) by including an inhibition/switching trial, the working memory and cognitive control networks best predict the score (mean predictive power = 0.36, 0.33), which is consistent with previous studies (Periáñez, 2021).
·Figure 1. Predictive performance and cognitive construct network.
·Figure 2. Predictive power of the six cognitive construct networks on 3/16 of the behavioral measures. Similar plots can be generated for the remaining 13/16 measures.
Conclusions:
This work's goal is to understand the contribution of each a priori-defined network to test scores, not to find the best predictive model as is usually done. Once we understand the brain systems that the tests rely upon, it is possible to better use tests in terms of interrogating specific brain networks. This opens avenues of research by providing a framework for developing measures guided by quantitative brain metrics.
Modeling and Analysis Methods:
Connectivity (eg. functional, effective, structural) 2
fMRI Connectivity and Network Modeling 1
Keywords:
Cognition
Computational Neuroscience
FUNCTIONAL MRI
MRI
Other - behavior
1|2Indicates the priority used for review
Abstract Information
By submitting your proposal, you grant permission for the Organization for Human Brain Mapping (OHBM) to distribute your work in any format, including video, audio print and electronic text through OHBM OnDemand, social media channels, the OHBM website, or other electronic publications and media.
The Open Science Special Interest Group (OSSIG) is introducing a reproducibility challenge for OHBM 2025. This new initiative aims to enhance the reproducibility of scientific results and foster collaborations between labs. Teams will consist of a “source” party and a “reproducing” party, and will be evaluated on the success of their replication, the openness of the source work, and additional deliverables. Click here for more information. Propose your OHBM abstract(s) as source work for future OHBM meetings by selecting one of the following options:
Please indicate below if your study was a "resting state" or "task-activation” study.
Healthy subjects only or patients (note that patient studies may also involve healthy subjects):
Was this research conducted in the United States?
Are you Internal Review Board (IRB) certified? Please note: Failure to have IRB, if applicable will lead to automatic rejection of abstract.
Were any human subjects research approved by the relevant Institutional Review Board or ethics panel? NOTE: Any human subjects studies without IRB approval will be automatically rejected.
Were any animal research approved by the relevant IACUC or other animal research panel? NOTE: Any animal studies without IACUC approval will be automatically rejected.
Please indicate which methods were used in your research:
For human MRI, what field strength scanner do you use?
Which processing packages did you use for your study?
Provide references using APA citation style.
Golden, C., Freshwater, S. M., & Golden, Z. (1978). Stroop color and word test.
Kaplan, E., Goodglass, H., & Weintraub, S. (2001). Boston naming test. The Clinical Neuropsychologist.
Lichtenberger, E. O. (2012). Essentials of WAIS-IV assessment (Vol. 22). John Wiley & Sons.
Periáñez, J. A., Lubrini, G., García-Gutiérrez, A., & Ríos-Lago, M. (2021). Construct validity of the stroop color-word test: influence of speed of visual search, verbal fluency, working memory, cognitive flexibility, and conflict monitoring. Archives of Clinical Neuropsychology, 36(1), 99-111.
Shen, X., Tokoglu, F., Papademetris, X., & Constable, R. T. (2013). Groupwise whole-brain parcellation from resting-state fMRI data for network node identification. Neuroimage, 82, 403-415.
Wechsler, D. (1999). Wechsler abbreviated scale of intelligence. Psychological Corporation.