Advancing Semantic Mapping for Naturalistic Settings with High-Density Diffuse Optical Tomography
Presented During: Poster Session 3
Friday, June 27, 2025: 01:45 PM - 03:45 PM
Friday, June 27, 2025: 01:45 PM - 03:45 PM
Presented During: Poster Session 4
Saturday, June 28, 2025: 01:45 PM - 03:45 PM
Saturday, June 28, 2025: 01:45 PM - 03:45 PM
Poster No:
1978
Submission Type:
Abstract Submission
Authors:
Wiete Fehner1, Morgan Fogarty1, AAHANA BAJRACHARYA1, Zachary Markow1, Dana Wilhelm1, Jerry Tang2, Jason Trobaugh1, Alexander Huth2, Joseph Culver1
Institutions:
1Washington University in St. Louis, St. Louis, MO, 2The University of Texas at Austin, Austin, TX
First Author:
Co-Author(s):
Introduction:
Visual semantic mapping represents a rich set of object category features that are connected to language processing and map vast cortical regions [4,6]. This method uses naturalistic stimuli and can be applied to explore language-related questions such as brain development and language disorders. Aphasia, a language disorder resulting from stroke-induced cerebral damage, affects ~200,000 US individuals annually, with recovery relying on brain plasticity [8]. Functional MRI (fMRI), the gold standard for semantic mapping, offers high spatial resolution but is constrained by immobility and cost, limiting its use in naturalistic settings. Conversely, sparse fNIRS systems are portable and suitable for naturalistic environments. However, they are limited to sparse and localized semantic category mapping as they lack the spatial resolution and Field of View (FOV) needed to map complex semantic spaces [7]. High-density diffuse optical tomography (HD-DOT) bridges this gap, allowing for naturalistic imaging like fNIRS and spatial resolution similar to fMRI [2,5]. This study demonstrates the feasibility of Very High-Density DOT (VHD-DOT), featuring ~10 mm optode spacing and ~10,000 measurement channels [3] for visual semantic mapping using naturalistic movie stimuli.
Methods:
Participants (N=5) completed three 89-minute VHD-DOT sessions, including localizers and movie clips (~4.5 h/participant). Sessions included 120 minutes of training and 90 minutes of repeated test movie data, labeled with 1708 semantic WordNet category labels from [4]. Cap placement followed a precision protocol for accurate co-registration between imaging sessions [1]. As a measure of SNR, explainable variance (EV) was used to assess the repeatability of naturalistic test movies, representing the portion of the signal variance related to the stimulus. We tested the feasibility of VHD-DOT for semantic category mapping. A 'person' regressor was created using binary labels, and the training data was divided into three subsets to assess the reproducibility of semantic category mapping (Fig 1A). A GLM was applied to these subsets to generate semantic maps for the 'person' category. The ratio of the mean and standard deviation across the subset of maps was computed to highlight statistically meaningful areas (Fig 1D). A linear voxelwise encoding model (VEM) utilizing all 1708 WordNet features was applied using regularized linear regression for each voxel, with bootstrapping for hyperparameter tuning. Prediction accuracy was computed as the correlation between the actual and predicted time traces. Principal Component Analysis (PCA) was applied to the derived category weight matrix for dimensionality reduction. This reduced representation allows the mapping of a complex semantic space. The training data was split in half to test the robustness of the PC maps. The model was trained on the first and second half of the data, and PC maps were generated for each weight matrix.
Results:
This study establishes VHD-DOT's feasibility for semantic mapping in naturalistic settings. EV maps for test movies showed high repeatability in visual areas (Fig 1B). The single regressor 'person' category maps showed reproducibility (Fig 1C-E). The 'person' category map derived from the VEM includes areas identified by the single regressor analysis, yet it extends more broadly, likely due to the model's increased complexity (Fig 1F). Additionally, accuracy maps validated the VEM (Fig 2A). The VEM derived semantic maps revealed voxel selectivity and a complex semantic space spanning broad cortical areas, with consistent PC map features across splits (Fig 2B-C).
Conclusions:
VHD-DOT bridges the gap between sparse fNIRS and fMRI, enabling semantic mapping in naturalistic contexts. This approach holds promise for studying language reorganization and plasticity in ecologically valid paradigms. Such capabilities are particularly valuable for exploring cortical reorganization in conditions like aphasia and other language disorders.
Language:
Language Comprehension and Semantics 2
Novel Imaging Acquisition Methods:
NIRS 1
Keywords:
Aphasia
Computational Neuroscience
Language
Near Infra-Red Spectroscopy (NIRS)
OPTICAL
Other - Semantics; HD-DOT; Novel Imaging Methods; Naturalistic Imaging
1|2Indicates the priority used for review
Abstract Information
By submitting your proposal, you grant permission for the Organization for Human Brain Mapping (OHBM) to distribute your work in any format, including video, audio print and electronic text through OHBM OnDemand, social media channels, the OHBM website, or other electronic publications and media.
The Open Science Special Interest Group (OSSIG) is introducing a reproducibility challenge for OHBM 2025. This new initiative aims to enhance the reproducibility of scientific results and foster collaborations between labs. Teams will consist of a “source” party and a “reproducing” party, and will be evaluated on the success of their replication, the openness of the source work, and additional deliverables. Click here for more information. Propose your OHBM abstract(s) as source work for future OHBM meetings by selecting one of the following options:
Please indicate below if your study was a "resting state" or "task-activation” study.
Healthy subjects only or patients (note that patient studies may also involve healthy subjects):
Was this research conducted in the United States?
Are you Internal Review Board (IRB) certified? Please note: Failure to have IRB, if applicable will lead to automatic rejection of abstract.
Were any human subjects research approved by the relevant Institutional Review Board or ethics panel? NOTE: Any human subjects studies without IRB approval will be automatically rejected.
Were any animal research approved by the relevant IACUC or other animal research panel? NOTE: Any animal studies without IACUC approval will be automatically rejected.
Please indicate which methods were used in your research:
For human MRI, what field strength scanner do you use?
Which processing packages did you use for your study?
Provide references using APA citation style.
[2] Eggebrecht, A. T. et al (2014). Mapping distributed brain function and networks with diffuse optical tomography. Nature Photonics, 8(6), 448–454. https://doi.org/10.1038/nphoton.2014.107
[3] Fogarty, M. et al (2023). A Whole Head Ultra-High Density Diffuse Optical Tomography System for Naturalistic and Resting State Functional Human Brain Mapping. https://doi.org/10.1364/brain.2023.bm2b.5
[4] Huth, A. G. et al (2012). A Continuous Semantic Space Describes the Representation of Thousands of Object and Action Categories across the Human Brain. Neuron, 76(6), 1210–1224. https://doi.org/10.1016/j.neuron.2012.10.014
[5] Markow, Z. E. et al (2023). Ultra-high density imaging arrays for diffuse optical tomography of human brain improve resolution, signal-to-noise, and information decoding. BioRxiv. https://doi.org/10.1101/2023.07.21.549920
[6] Popham, S. F. et al (2021). Visual and linguistic semantic representations are aligned at the border of human visual cortex. Nature Neuroscience, 24(11), 1628–1636. https://doi.org/10.1038/s41593-021-00921-6
[7] Rybář, M. et al (2022). Neural decoding of semantic concepts: a systematic literature review. Journal of Neural Engineering, 19(2), 021002. https://doi.org/10.1088/1741-2552/ac619a
[8] Stefaniak, J. D. et al (2019). The neural and neurocomputational bases of recovery from post-stroke aphasia. Nature Reviews Neurology, 16(1), 43–55. https://doi.org/10.1038/s41582-019-0282-1