A guide for image selection, tools, and methods for image-based meta-analysis with NeuroVault
Presented During: Poster Session 3
Friday, June 27, 2025: 01:45 PM - 03:45 PM
Friday, June 27, 2025: 01:45 PM - 03:45 PM
Presented During: Poster Session 4
Saturday, June 28, 2025: 01:45 PM - 03:45 PM
Saturday, June 28, 2025: 01:45 PM - 03:45 PM
Poster No:
1543
Submission Type:
Abstract Submission
Authors:
Julio Peraza1, James Kent2, Ross Blair3, Jean-Baptiste Poline4, Thomas Nichols5, Alejandro de la Vega6, Angela Laird1
Institutions:
1Florida International University, Miami, FL, 2UT Austin, Austin, TX, 3Stanford University, Stanford, CA, 4McGill University, Montreal, Quebec, 5University of Oxford, Oxford, Oxfordshire, 6University of Texas at Austin, Austin, TX
First Author:
Co-Author(s):
Introduction:
Despite the increasing adoption of the NeuroVault repository in the neuroimaging community, downloading usable data for meta-analyses still comes with multiple challenges. For example, a substantial portion of collections in NeuroVault are annotated incorrectly or lack a link to a valid publication; some images are duplicates, while others correspond to non-statistical imaging modalities. Overall, the potential number of spurious statistical maps complicates the use of NeuroVault data for image-based meta-analyses (IBMA). Here, we explored the current state of NeuroVault, identified different challenges in the data, and proposed an automatic heuristic selection framework for curating the data to perform IBMA.
Methods:
Figure 1 provides an overview of our methodological approach. First, we identified fRMI tasks linked to a specific cognitive domain (e.g., working memory, motor, and emotion processing) using the already established connections in the Cognitive Atlas knowledge base. Then, we downloaded all images linked to the selected tasks from the NeuroVault repository. Second, we performed a preliminary image selection leveraging the metadata associated with the images, which were selected as potential candidates for IBMA. Next, we conducted a data-driven heuristic selection to remove possible outliers from the data. Subsequently, we manually selected relevant images by identifying the analysis contrast in their corresponding article with the help of the image metadata in NeuroVault. Third, we conducted image-based meta-analyses using standardized effect size maps of the chosen images. In addition to using a baseline estimator (i.e., Mean), we explore four more robust combination methods: median, trim mean, winsorized mean, and weighted mean. Finally, the meta-analyses with different combinations of parameters (i.e., image selection method and combination approach) were evaluated against reference images from the task-fMRI group-average effect size maps from the Human Connectome Project (HCP) S1200 data release.
Results:
We extensively analyze the state of the NeuroVault data as of February 2024. We discovered a wide representation of task and cognitive domains, providing enough data to run image-based meta-analyses. A preliminary selection of images, followed by a data-driven heuristic selection and finished with a manual selection, yielded ten images from six collections for "working memory," 30 images from seven collections for "motor," and eight images from only two collections for "emotion processing." The manual meta-analysis reproduced the reference maps for the three cognitive domains in question, showing that the NeuroVault data is amenable to IBMA (Figure 2A). The manual meta-analysis also delivered the highest correlation coefficient with the reference maps relative to the other selection methods (Figure 2B). Heuristic selection leads to better correlation coefficients for working memory (i.e., 0.65) and emotion processing (0.25) than including all images. This result indicates that our selection procedure helped reduce extreme and unwanted values from all image samples.
Then, we compared five different meta-analytic estimators (Figure 2C). The estimators perform similarly for less heterogeneous cognitive domains such as working memory and motor. For more complex cognitive domains like emotion processing, a more robust estimator makes a substantial difference compared to the baseline estimator. Surprisingly, when including all images, where we initially find a correlation of 0.05 for the baseline, using the median as a combination method improves the correlation to a 0.2 value matching the result of the heuristic selection with the baseline estimator.
Then, we compared five different meta-analytic estimators (Figure 2C). The estimators perform similarly for less heterogeneous cognitive domains such as working memory and motor. For more complex cognitive domains like emotion processing, a more robust estimator makes a substantial difference compared to the baseline estimator. Surprisingly, when including all images, where we initially find a correlation of 0.05 for the baseline, using the median as a combination method improves the correlation to a 0.2 value matching the result of the heuristic selection with the baseline estimator.
Conclusions:
Taken together, the results of this work indicate that IBMA is feasible with Neurovault data. Our proposed image selection framework, supplemented with openly available tools and reproducible methods, encourages best practices and guides future IBMA applications with NeuroVault.
Modeling and Analysis Methods:
Methods Development 1
Univariate Modeling
Other Methods
Neuroinformatics and Data Sharing:
Workflows 2
Informatics Other
Keywords:
Design and Analysis
FUNCTIONAL MRI
Meta- Analysis
Workflows
Other - image-based meta-analysis
1|2Indicates the priority used for review
Abstract Information
By submitting your proposal, you grant permission for the Organization for Human Brain Mapping (OHBM) to distribute your work in any format, including video, audio print and electronic text through OHBM OnDemand, social media channels, the OHBM website, or other electronic publications and media.
The Open Science Special Interest Group (OSSIG) is introducing a reproducibility challenge for OHBM 2025. This new initiative aims to enhance the reproducibility of scientific results and foster collaborations between labs. Teams will consist of a “source” party and a “reproducing” party, and will be evaluated on the success of their replication, the openness of the source work, and additional deliverables. Click here for more information. Propose your OHBM abstract(s) as source work for future OHBM meetings by selecting one of the following options:
Please indicate below if your study was a "resting state" or "task-activation” study.
Healthy subjects only or patients (note that patient studies may also involve healthy subjects):
Was this research conducted in the United States?
Are you Internal Review Board (IRB) certified? Please note: Failure to have IRB, if applicable will lead to automatic rejection of abstract.
Were any human subjects research approved by the relevant Institutional Review Board or ethics panel? NOTE: Any human subjects studies without IRB approval will be automatically rejected.
Were any animal research approved by the relevant IACUC or other animal research panel? NOTE: Any animal studies without IACUC approval will be automatically rejected.
Please indicate which methods were used in your research:
For human MRI, what field strength scanner do you use?
Which processing packages did you use for your study?
Provide references using APA citation style.
Freedman, D., Lane, D., 1983. A Nonstochastic Interpretation of Reported Significance Levels. Journal of Business & Economic Statistics 1, 292–298. https://doi.org/10.2307/1391660
Gorgolewski, K.J., Varoquaux, G., Rivera, G., Schwarz, Y., Ghosh, S.S., Maumet, C., Sochat, V.V., Nichols, T.E., Poldrack, R.A., Poline, J.-B., Yarkoni, T., Margulies, D.S., 2015. NeuroVault.org: a web-based repository for collecting and sharing unthresholded statistical maps of the human brain. Frontiers in Neuroinformatics 9, 8. https://doi.org/10.3389/fninf.2015.00008
Menuet, R., Meudec, R., Dockès, J., Varoquaux, G., Thirion, B., 2022. Comprehensive decoding mental processes from Web repositories of functional brain images. Sci Rep 12, 7050. https://doi.org/10.1038/s41598-022-10710-1
Poldrack, R.A., Kittur, A., Kalar, D., Miller, E., Seppa, C., Gil, Y., Parker, D.S., Sabb, F.W., Bilder, R.M., 2011. The cognitive atlas: toward a knowledge foundation for cognitive neuroscience. Front Neuroinform 5, 17. https://doi.org/10.3389/fninf.2011.00017
Salimi-Khorshidi, G., Smith, S.M., Keltner, J.R., Wager, T.D., Nichols, T.E., 2009. Meta-analysis of neuroimaging data: A comparison of image-based and coordinate-based pooling of studies. NeuroImage 45, 810–823. https://doi.org/10.1016/j.neuroimage.2008.12.039
Salo, T., Yarkoni, T., Nichols, T.E., Poline, J.-B., Bilgel, M., Bottenhorn, K.L., Jarecka, D., Kent, J.D., Kimbler, A., Nielson, D.M., Oudyk, K.M., Peraza, J.A., Pérez, A., Reeders, P.C., Yanes, J.A., Laird, A.R., 2023. NiMARE: Neuroimaging Meta-Analysis Research Environment. Aperture Neuro 3, 1–32. https://doi.org/10.52294/001c.87681