Physics-Informed Deep Learning Approaches to Restore and Enhance Hyperspectral Images of Brain
Presented During:
Friday, June 27, 2025: 11:30 AM - 12:45 PM
Brisbane Convention & Exhibition Centre
Room:
M4 (Mezzanine Level)
Poster No:
1932
Submission Type:
Abstract Submission
Authors:
Yuchen Xiang1, Zhaolu Liu1, Monica Garcia-Segura1, Daniel Simon1, Boxuan Cao1, Vincen Wu1, Kenneth Robinson1, Ronan Battle1, Yu Wang1, Robert Murray1, Luca Peruzzotti Jametti2, Zoltan Takats1
Institutions:
1Imperial College London, London, UK, 2University of Cambridge, Cambridge, UK
First Author:
Co-Author(s):
Introduction:
Hyperspectral imaging (HSI) is vital in biomedical fields like brain imaging, distinguishing tissues via spectral differences for diagnostics and surgical tools (Fabelo, 2018; Yoon, 2022). Mass Spectrometry Imaging (MSI), a prominent HSI technique, maps thousands of chemicals but faces "3S triangle" trade-offs between spectral resolution, spatial resolution, and speed, limiting spatial resolution to 10–100 µm.
HyReS, a deep learning-based restoration and super-resolution method, overcomes these challenges. By integrating physics-informed Fourier constraints, it ensures spectral and spatial fidelity without large datasets. HyReS restores resolution beyond hardware limits, preserving data integrity and enabling precise downstream analyses, advancing HSI applications.
HyReS, a deep learning-based restoration and super-resolution method, overcomes these challenges. By integrating physics-informed Fourier constraints, it ensures spectral and spatial fidelity without large datasets. HyReS restores resolution beyond hardware limits, preserving data integrity and enabling precise downstream analyses, advancing HSI applications.
Methods:
To address limited training data, paired datasets are synthetically generated by cropping images to dimensions divisible by the scaling factor (4), downsampling them with a bicubic kernel, and adding adaptive noise to create low-resolution (LR) counterparts 16× smaller than the originals.
HyReS uses a GAN-based SISR architecture (Real-ESRGAN) (Wang, 2021) with a Generator for upscaling LR images and a Discriminator for iterative quality improvement. Unlike standard models, HyReS incorporates a novel Fourier Ring Correlation-based loss function (LFRC) to enhance resolution in the frequency domain. Training was optimized using Adam with 100 epochs, batch size 8, and balanced loss functions for the Generator (LFRC) and Discriminator (Cross Entropy).
Quantitative assessment of perceptual quality and resolution combines CRISQUE, a no-reference metric integrating BRISQUE and PIQE, with FRC for resolution estimation. This ensures both perceptual quality and resolution improvements in restored images.
HyReS uses a GAN-based SISR architecture (Real-ESRGAN) (Wang, 2021) with a Generator for upscaling LR images and a Discriminator for iterative quality improvement. Unlike standard models, HyReS incorporates a novel Fourier Ring Correlation-based loss function (LFRC) to enhance resolution in the frequency domain. Training was optimized using Adam with 100 epochs, batch size 8, and balanced loss functions for the Generator (LFRC) and Discriminator (Cross Entropy).
Quantitative assessment of perceptual quality and resolution combines CRISQUE, a no-reference metric integrating BRISQUE and PIQE, with FRC for resolution estimation. This ensures both perceptual quality and resolution improvements in restored images.
Results:
HyReS accelerates experimental MSI
by improving imaging speed without sacrificing resolution. Using HR (25 µm) and LR (100 µm) mouse brain images for training and testing, HyReS restored LR inputs with enhanced resolution and SNR (Fig. 1), addressing challenges like SNR loss and artefacts in HR imaging.
Quantitatively, CRISQUE scores rose from 43.23 (LR) to 46.88 (HyReS), and resolution improved from 531.4 µm (LR) to 87.2 µm (HyReS), outperforming HR imaging (212.5 µm). Imaging time dropped from 6 hours (HR) to 33 minutes (LR + HyReS), enabling high-throughput applications like digital pathology.
HyReS preserves biological context in MSI data.
In an MS dataset, metabolic differences between healthy and diseased brains were maintained after restoration. Classification accuracy lost to downsampling (~15%) was nearly fully restored (Fig. 2b), with minimal artefacts and high spectral fidelity (Spearman correlation: 0.9982, Fig. 2d). Despite minor hallucination effects, HyReS enhances MSI quality and preserves biological information, demonstrating its potential for biomedical applications.
by improving imaging speed without sacrificing resolution. Using HR (25 µm) and LR (100 µm) mouse brain images for training and testing, HyReS restored LR inputs with enhanced resolution and SNR (Fig. 1), addressing challenges like SNR loss and artefacts in HR imaging.
Quantitatively, CRISQUE scores rose from 43.23 (LR) to 46.88 (HyReS), and resolution improved from 531.4 µm (LR) to 87.2 µm (HyReS), outperforming HR imaging (212.5 µm). Imaging time dropped from 6 hours (HR) to 33 minutes (LR + HyReS), enabling high-throughput applications like digital pathology.
HyReS preserves biological context in MSI data.
In an MS dataset, metabolic differences between healthy and diseased brains were maintained after restoration. Classification accuracy lost to downsampling (~15%) was nearly fully restored (Fig. 2b), with minimal artefacts and high spectral fidelity (Spearman correlation: 0.9982, Fig. 2d). Despite minor hallucination effects, HyReS enhances MSI quality and preserves biological information, demonstrating its potential for biomedical applications.
Conclusions:
HyReS networks trained with the FRCGAN framework can be applied to diverse biomedical applications without extensive training data. By leveraging inter-data correlations and imaging models, HyReS overcomes HSI's "3S" constraints. For instance, brain tissue sections restored from 4× downsampled inputs maintained resolution and quality.
HyReS enables high-throughput applications like histopathology, increasing imaging speed by 16× for MSI while preserving spatial resolution. It also enhances downstream analyses by denoising and restoring features lost to under-sampling or noise, supporting accurate machine learning tasks without compromising biological information.
Validation in EAE mouse models preserved metabolic profiles and key molecular distinctions, demonstrating its utility for biologically precise studies. Future integration with advanced imaging hardware could unlock multimodal studies across scales, particularly in brain sciences.
HyReS enables high-throughput applications like histopathology, increasing imaging speed by 16× for MSI while preserving spatial resolution. It also enhances downstream analyses by denoising and restoring features lost to under-sampling or noise, supporting accurate machine learning tasks without compromising biological information.
Validation in EAE mouse models preserved metabolic profiles and key molecular distinctions, demonstrating its utility for biologically precise studies. Future integration with advanced imaging hardware could unlock multimodal studies across scales, particularly in brain sciences.
Modeling and Analysis Methods:
Methods Development
Multivariate Approaches
Novel Imaging Acquisition Methods:
Multi-Modal Imaging 2
Imaging Methods Other 1
Physiology, Metabolism and Neurotransmission:
Cerebral Metabolism and Hemodynamics
Keywords:
Machine Learning
Open-Source Software
Other - mass spectrometry imaging, deep learning, super-resolution
1|2Indicates the priority used for review
Abstract Information
By submitting your proposal, you grant permission for the Organization for Human Brain Mapping (OHBM) to distribute your work in any format, including video, audio print and electronic text through OHBM OnDemand, social media channels, the OHBM website, or other electronic publications and media.
The Open Science Special Interest Group (OSSIG) is introducing a reproducibility challenge for OHBM 2025. This new initiative aims to enhance the reproducibility of scientific results and foster collaborations between labs. Teams will consist of a “source” party and a “reproducing” party, and will be evaluated on the success of their replication, the openness of the source work, and additional deliverables. Click here for more information. Propose your OHBM abstract(s) as source work for future OHBM meetings by selecting one of the following options:
Please indicate below if your study was a "resting state" or "task-activation” study.
Healthy subjects only or patients (note that patient studies may also involve healthy subjects):
Was this research conducted in the United States?
Were any human subjects research approved by the relevant Institutional Review Board or ethics panel? NOTE: Any human subjects studies without IRB approval will be automatically rejected.
Were any animal research approved by the relevant IACUC or other animal research panel? NOTE: Any animal studies without IACUC approval will be automatically rejected.
Please indicate which methods were used in your research:
Provide references using APA citation style.
[2] Jonghee Yoon (2022). Hyperspectral Imaging for Clinical Applications. BioChip Journal, 16(1):1–12.
[3] Xintao Wang et al (2021). Real-ESRGAN: Training Real-World Blind Super- Resolution with Pure Synthetic Data. arXiv:2107.10833.