Multimodal and Multiscale Whole-Brain Reconstruction of the Monkey Brain
Poster No:
1855
Submission Type:
Abstract Submission
Authors:
Bokai Zhao1,2, Dong Zhenwei3,2, Zhengyi Yang4,2, Ming Song4,2, Tianzi Jiang3,2,5
Institutions:
1School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing ,China,, 2Brainnetome Center, Institute of Automation, Chinese Academy of Sciences, Beijing, China, 3School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China, 4School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, Beijing, 5Xiaoxiang Institute for Brain Health and Yongzhou Central Hospital, Yongzhou , China
First Author:
Bokai Zhao
School of Artificial Intelligence, University of Chinese Academy of Sciences|Brainnetome Center, Institute of Automation, Chinese Academy of Sciences
Beijing ,China,|Beijing, China
School of Artificial Intelligence, University of Chinese Academy of Sciences|Brainnetome Center, Institute of Automation, Chinese Academy of Sciences
Beijing ,China,|Beijing, China
Co-Author(s):
Dong Zhenwei
School of Artificial Intelligence, University of Chinese Academy of Sciences|Brainnetome Center, Institute of Automation, Chinese Academy of Sciences
Beijing, China|Beijing, China
School of Artificial Intelligence, University of Chinese Academy of Sciences|Brainnetome Center, Institute of Automation, Chinese Academy of Sciences
Beijing, China|Beijing, China
Zhengyi Yang
School of Artificial Intelligence, University of Chinese Academy of Sciences|Brainnetome Center, Institute of Automation, Chinese Academy of Sciences
Beijing, Beijing|Beijing, China
School of Artificial Intelligence, University of Chinese Academy of Sciences|Brainnetome Center, Institute of Automation, Chinese Academy of Sciences
Beijing, Beijing|Beijing, China
Ming Song
School of Artificial Intelligence, University of Chinese Academy of Sciences|Brainnetome Center, Institute of Automation, Chinese Academy of Sciences
Beijing, Beijing|Beijing, China
School of Artificial Intelligence, University of Chinese Academy of Sciences|Brainnetome Center, Institute of Automation, Chinese Academy of Sciences
Beijing, Beijing|Beijing, China
Tianzi Jiang
School of Artificial Intelligence, University of Chinese Academy of Sciences|Brainnetome Center, Institute of Automation, Chinese Academy of Sciences|Xiaoxiang Institute for Brain Health and Yongzhou Central Hospital
Beijing, China|Beijing, China|Yongzhou , China
School of Artificial Intelligence, University of Chinese Academy of Sciences|Brainnetome Center, Institute of Automation, Chinese Academy of Sciences|Xiaoxiang Institute for Brain Health and Yongzhou Central Hospital
Beijing, China|Beijing, China|Yongzhou , China
Introduction:
In neuroscience research, integrating the brain's multi-scale features poses a significant challenge(Amunts, 2013). Although magnetic resonance imaging (MRI) enables the non-invasive or ex vivo acquisition of macroscopic brain structure images, more detailed characterization still relies on histological procedures and microscopic analysis (Mancini, 2020). To address this, we have collected and constructed a multi-modal, cross-scale macaque brain imaging dataset and established a comprehensive workflow for data processing and reconstruction, spanning from 2D slices to 3D high-resolution brain structures.
Methods:
Histology provides exceptional contrast at the microscopic scale. however, as a 2D modality, the slicing and staining processes inevitably induce tissue distortions. In contrast, MRI despite its lower resolution, can generate undistorted 3D images of the brain. Consequently, integrating these two modalities offers an effective solution for high-resolution 3D brain imaging.Through the material processing workflow shown in Figure 1A, we obtained MRI scans of the macaque brain, 347 stained slices, and 2,000 blockface images.The microscopic images are segmented at a resolution of 1 μm using the OTSU(Otsu, 1975) algorithm to create masks. The SAM (Kirillov, 2023) annotation tool is employed to label brain regions in the corresponding blockface images of the stained slices, and the annotated data are used to train a UNet (Ronneberger, 2015) model for segmenting the remaining images. MRI images are segmented by means of OTSU(Otsu, 1975) thresholding combined with morphological processing techniques. VoxelMorph(Balakrishnan, 2019) is employed for deformable registration to effect the alignment between the downsampled microscopic stained images and their corresponding blockface images. For the segmented blockface images, rigid registration is sequentially applied from the central image outward to both ends, thereby constructing a 3D brain model(Pichat, 2018). Finally, the 3D brain model is deformably registered with the MRI images, and an invertible transformation matrix is employed to map MRI voxels to histological regions.
Results:
We constructed a multimodal, cross-scale macaque brain imaging dataset, integrating high-resolution histological data with undistorted three-dimensional MRI images. The image segmentation and registration processes achieved precise alignment between histological slices and MRI data, as demonstrated in Figure 2. The resulting 3D brain model accurately reflects the macaque brain's structural details.
Conclusions:
Our study successfully integrated histological and MRI data using AI-driven image segmentation and registration techniques, enabling precise spatial mapping of the macaque brain's multi-scale architecture. By aligning histological slices with MRI through blockface images, we reconstructed high-resolution 3D brain structures, effectively tackling the challenge of multi-scale feature integration. This approach facilitates the combined analysis of spatial and imaging omics(Bressan, 2023), providing a robust solution for high-resolution brain imaging and advancing our understanding of the brain's complex structures and functions.In the future, we aim to publicly release our data to accelerate progress in spatial omics and imaging omics.
Modeling and Analysis Methods:
Image Registration and Computational Anatomy 2
Methods Development
Neuroinformatics and Data Sharing:
Databasing and Data Sharing
Workflows 1
Novel Imaging Acquisition Methods:
Multi-Modal Imaging
Keywords:
Computing
Data Registration
MRI
Open Data
Spatial Warping
Workflows
1|2Indicates the priority used for review
Abstract Information
By submitting your proposal, you grant permission for the Organization for Human Brain Mapping (OHBM) to distribute your work in any format, including video, audio print and electronic text through OHBM OnDemand, social media channels, the OHBM website, or other electronic publications and media.
The Open Science Special Interest Group (OSSIG) is introducing a reproducibility challenge for OHBM 2025. This new initiative aims to enhance the reproducibility of scientific results and foster collaborations between labs. Teams will consist of a “source” party and a “reproducing” party, and will be evaluated on the success of their replication, the openness of the source work, and additional deliverables. Click here for more information. Propose your OHBM abstract(s) as source work for future OHBM meetings by selecting one of the following options:
Please indicate below if your study was a "resting state" or "task-activation” study.
Healthy subjects only or patients (note that patient studies may also involve healthy subjects):
Was this research conducted in the United States?
Were any human subjects research approved by the relevant Institutional Review Board or ethics panel? NOTE: Any human subjects studies without IRB approval will be automatically rejected.
Were any animal research approved by the relevant IACUC or other animal research panel? NOTE: Any animal studies without IACUC approval will be automatically rejected.
Please indicate which methods were used in your research:
For human MRI, what field strength scanner do you use?
Which processing packages did you use for your study?
Provide references using APA citation style.
2.Pichat, J., Iglesias, J. E., Yousry, T., Ourselin, S., & Modat, M. (2018). A survey of methods for 3D histology reconstruction. Medical image analysis, 46, 73-105.
3.Mancini, M., Casamitjana, A., Peter, L., Robinson, E., Crampsie, S., Thomas, D. L., ... & Iglesias, J. E. (2020). A multimodal computational pipeline for 3D histology of the human brain. Scientific reports, 10(1), 13839.
4.Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., ... & Girshick, R. (2023). Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 4015-4026).
5.Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18 (pp. 234-241). Springer International Publishing.
6.Balakrishnan, G., Zhao, A., Sabuncu, M. R., Guttag, J., & Dalca, A. V. (2019). Voxelmorph: a learning framework for deformable medical image registration. IEEE transactions on medical imaging, 38(8), 1788-1800.
7.Otsu, N. (1975). A threshold selection method from gray-level histograms. Automatica, 11(285-296), 23-27.
8.Bressan, D., Battistoni, G., & Hannon, G. J. (2023). The dawn of spatial omics. Science, 381(6657), eabq4964.
9.Avants, B. B., Tustison, N., & Song, G. (2009). Advanced normalization tools (ANTS). Insight j, 2(365), 1-35.