Braincoder: A Python library for fitting and inverting encoding models for fMRI
Poster No:
1535
Submission Type:
Abstract Submission
Authors:
Gilles de Hollander1, Maike Renkert1, Tomas Knapen2, Christian Ruff1
Institutions:
1University of Zurich, Zurich, Zurich, 2Vrije Universiteit Amsterdam, Amsterdam, NH
First Author:
Co-Author(s):
Introduction:
Understanding how the human brain represents the outside world is one of the central challenges in cognitive neuroscience. Encoding models for functional MRI (fMRI) offer a powerful approach to this question, allowing researchers to estimate the relationship between objective stimulus properties and neural activity measured by single-voxel blood oxygenation level-dependent (BOLD) responses (Naselaris et al., 2011). These models have been instrumental in determining neural representations of sensory features such as orientation (Kay et al., 2008), visual space (Dumoulin & Wandell, 2008), numerosity (Harvey et al., 2013), and auditory frequency (Moerel et al., 2013). A recent breakthrough in this field is the ability to invert encoding models, making it possible to decode stimulus features in a mathematically principled way. Bayesian inversion (van Bergen et al., 2015), in particular, offers unique advantages by explicitly modeling the likelihood of stimuli given observed neural activity, allowing not only the decoding of the maximally likely (ML) stimulus but also the mean posterior estimate and its uncertainty. Furthermore, it allows researchers to quantify the precision of encoding across stimulus space using principled information theoretic measures like Fisher information. Thus, these tools can provide insights into how specific patterns of neural activity dynamically prioritize encoding of certain stimulus features, with implications for cognitive theory. However, the complexity of implementing encoding models has hindered their broader adoption in human neuroimaging.
Methods:
Braincoder is a toolbox that allows researchers to flexibly fit various encoding models. Its architecture is highly object-oriented, which makes it relatively easy to extend existing or implement new models. For this latter purpose, researchers just have to subclass one EncodingModel- and possibly one Stimulus-class. The estimation and Bayesian inversion of the model function automatically. The library also includes tools to estimate mean stimulus posterior, its dispersion, and Fisher information. It can gracefully deal with multidimensional stimulus spaces such as visual (i.e. x/y) or pixel space.
Results:
Braincoder provides implementations of widely used models, including 1D Gaussian/Von Mises receptive field models for features like numerosity and orientation, and 2D models for retinotopic mapping, such as the population receptive field (pRF), difference-of-Gaussians, and divisive normalization models (Aqil et al., 2021). Its versatility has been demonstrated in ongoing studies, such as decoding the parietal approximate number system (Barretto-García et al., 2023), exploring gaze-dependent retinotopic coding in the visual cortex (Szinte et al., 2024), and investigating value representations in the ventromedial prefrontal cortex (vmPFC). These studies highlight the potential of Braincoder to advance theoretical and empirical work in cognitive neuroscience. Additionally, example datasets and tutorials on https://braincoder-devs.github.io/ support researchers in learning and applying these tools.
·Short code example of how to fit a visual Gaussian receptive field model to data provided with the package.
Conclusions:
Braincoder represents a significant step forward in the practical application of encoding models for fMRI and their Bayesian inversion. By combining a user-friendly API with high computational efficiency, it lowers the barrier to entry for researchers and enables the exploration of new questions about how the brain represents and prioritizes information. With its comprehensive documentation, open-source design, and applicability to a range of neural encoding and decoding challenges, Braincoder should become a valuable resource for the cognitive neuroscience community.
Modeling and Analysis Methods:
Methods Development 1
Multivariate Approaches 2
Perception, Attention and Motor Behavior:
Perception: Visual
Keywords:
Multivariate
Open Data
Open-Source Code
Open-Source Software
Perception
Vision
1|2Indicates the priority used for review
Abstract Information
By submitting your proposal, you grant permission for the Organization for Human Brain Mapping (OHBM) to distribute your work in any format, including video, audio print and electronic text through OHBM OnDemand, social media channels, the OHBM website, or other electronic publications and media.
The Open Science Special Interest Group (OSSIG) is introducing a reproducibility challenge for OHBM 2025. This new initiative aims to enhance the reproducibility of scientific results and foster collaborations between labs. Teams will consist of a “source” party and a “reproducing” party, and will be evaluated on the success of their replication, the openness of the source work, and additional deliverables. Click here for more information. Propose your OHBM abstract(s) as source work for future OHBM meetings by selecting one of the following options:
Please indicate below if your study was a "resting state" or "task-activation” study.
Healthy subjects only or patients (note that patient studies may also involve healthy subjects):
Was this research conducted in the United States?
Were any human subjects research approved by the relevant Institutional Review Board or ethics panel? NOTE: Any human subjects studies without IRB approval will be automatically rejected.
Were any animal research approved by the relevant IACUC or other animal research panel? NOTE: Any animal studies without IACUC approval will be automatically rejected.
Please indicate which methods were used in your research:
For human MRI, what field strength scanner do you use?
Which processing packages did you use for your study?
Provide references using APA citation style.
Barretto-García, M., de Hollander, G., Grueschow, M., Polanía, R., Woodford, M., & Ruff, C. C. (2023). Individual risk attitudes arise from noise in neurocognitive magnitude representations. Nature Human Behaviour, 7(9), 1551–1567. https://doi.org/10.1038/s41562-023-01643-4
Dumoulin, S. O., & Wandell, B. A. (2008). Population receptive field estimates in human visual cortex. 39(2). https://doi.org/10.1016/j.neuroimage.2007.09.034
Harvey, B. M., Klein, B. P., Petridou, N., & Dumoulin, S. O. (2013). Topographic Representation of Numerosity in the Human Parietal Cortex. Science, 341(6150), 1123–1126. https://doi.org/10.1126/science.1239052
Kay, K. N., Naselaris, T., Prenger, R. J., & Gallant, J. L. (2008). Identifying natural images from human brain activity. 452(7185). https://doi.org/10.1038/nature06713
Moerel, M., Martino, D. F., Santoro, R., Ugurbil, K., Goebel, R., Yacoub, E., & Formisano, E. (2013). Processing of Natural Sounds: Characterization of Multipeak Spectral Tuning in Human Auditory Cortex. 33(29). https://doi.org/10.1523/jneurosci.5306-12.2013
Naselaris, T., Kay, K. N., Nishimoto, S., & Gallant, J. L. (2011). Encoding and decoding in fMRI. NeuroImage, 56(2), 400–410. https://doi.org/10.1016/j.neuroimage.2010.07.073
Sprague, T. C., & Serences, J. T. (2013). Attention modulates spatial priority maps in the human occipital, parietal and frontal cortices. Nature Neuroscience, 16(12), 1879–1887. https://doi.org/10.1038/nn.3574
Szinte, M., de Hollander, G., Aqil, M., Veríssimo, I., Dumoulin, S., & Knapen, T. (2024). A retinotopic reference frame for space throughout human visual cortex. BioRxiv, 2024.02.05.578862. https://doi.org/10.1101/2024.02.05.578862
van Bergen, R. S., Ma, W. J., Pratte, M. S., & Jehee, J. F. M. (2015). Sensory uncertainty decoded from visual cortex predicts behavior. Nature Neuroscience, 18(12), 1728. https://doi.org/10.1038/nn.4150