Precision ICU Resource Planning: A Multimodal Model for Brain Surgery Outcomes
Poster No:
1969
Submission Type:
Abstract Submission
Authors:
Maximilian Fischer1, Robin Peretzke1, Florian Hauptmann1, Peter Neher1, Paul Naser2, Jan-Oliver Neumann2, Klaus Maier-Hein1
Institutions:
1German Cancer Research Center, Heidelberg, Baden-Württemberg, 2Heidelberg University, Heidelberg, Baden-Württemberg
First Author:
Co-Author(s):
Introduction:
Advanced neurosurgical techniques have reduced Intensive Care Unit (ICU) admissions after brain surgeries, even for elderly patients [1]. Recent studies have shown great potential in predicting the occurrence of events that require ICU admission [2, 3]. However these studies rely only on clinical data and neglect imaging data. Imaging data captures important features in the tumor microenvironment that potentially affects post-surgery ICU admission, which can not be modelled by clinical data. In this work, we aim to improve ICU admission predictions by incorporating imaging data via a brain foundation model. Brain foundation models have recently shown great results in providing rich and low-dimensional image features for subsequent classification or segmentation tasks.
Methods:
Previous work has shown that clinical tabular data provides a good data basis for predicting the occurrence of ICU-events [3]. In this work we combine tabular clinical data with imaging features. Our dataset consists of a total of 611 subjects which is divided into 552 ICU-negative and 59 ICU-positive subjects. For each subject clinical data as well as a tumor segmentation mask is available. The clinical data consists of two parts: (i) data that is available already pre-surgery such as age or gender and (ii) data that is only available post-surgery such as surgery duration. The exact clinical parameters that we used for our study can be found in [3]. As classification models, we compared the baseline Gradient Boosted Tree from [3] against a more recent ResNet and the multimodal DAFT [4] model. To create image embeddings, we used a pre-trained Auto Encoder model that was trained on our dataset to extract embeddings from the 2D slice that contains the largest tumor diameter, which we determined via the segmentation mask. We refer to this approach as 2D latent. Additionally, we used the integrated DAFT 3D encoder (we refer to these embeddings as 3D ROI) and the foundation model [5] to create embeddings from the 3D image volume (3D SSL). All embeddings are created independently from training the classification model and for the classification task we trained a five-fold cross-validation and used the F1 score to determine the model performance, due to the highly imbalanced dataset. In our experiments, we especially evaluate the difference between using pre- or post-operative data in order to maximize clinical uptake of our findings.
Results:
As it is shown in the results figure, with our baseline the Gradient Boosted Tree, we achieved a performance on preoperative tabular data of [0.288 ± 0.059] and for pre- and postoperative tabular data [0.368 ± 0.075]. With the proposed combination of rich brain features extracted with the foundation model and the clinical data via DAFT, we could improve this score with only preoperative data to [0.297 ± 0.095] and for pre- and postoperative data to [0.407 ± 0.077]. Our results also show that now relevant image features are captured by a 2D image encoder, as indicated by the poor performance of all approaches when 2D latent data is used.
Conclusions:
With our experiments, we have shown that large-scale pre-trained models show great potential in extracting meaningful image features from brain MRIs. Advanced fusion strategies like DAFT also combine tabular clinical data efficiently with imaging data in order to improve ICU-admission prediction. In conclusion, our experiments show that ICU-admission prediction is not a task that can be solved solely based on a single modality. However, combining different modalities can help to improve the classification performance. Moreover, large pretrained models as feature extractors are beneficial especially for heavy class imbalanced cohorts. Insights from this work can streamline future experiments to achieve clinical relevant results.
Modeling and Analysis Methods:
Methods Development 2
Novel Imaging Acquisition Methods:
Multi-Modal Imaging 1
Keywords:
MRI
1|2Indicates the priority used for review
Abstract Information
By submitting your proposal, you grant permission for the Organization for Human Brain Mapping (OHBM) to distribute your work in any format, including video, audio print and electronic text through OHBM OnDemand, social media channels, the OHBM website, or other electronic publications and media.
The Open Science Special Interest Group (OSSIG) is introducing a reproducibility challenge for OHBM 2025. This new initiative aims to enhance the reproducibility of scientific results and foster collaborations between labs. Teams will consist of a “source” party and a “reproducing” party, and will be evaluated on the success of their replication, the openness of the source work, and additional deliverables. Click here for more information. Propose your OHBM abstract(s) as source work for future OHBM meetings by selecting one of the following options:
Please indicate below if your study was a "resting state" or "task-activation” study.
Healthy subjects only or patients (note that patient studies may also involve healthy subjects):
Was this research conducted in the United States?
Were any human subjects research approved by the relevant Institutional Review Board or ethics panel? NOTE: Any human subjects studies without IRB approval will be automatically rejected.
Were any animal research approved by the relevant IACUC or other animal research panel? NOTE: Any animal studies without IACUC approval will be automatically rejected.
Please indicate which methods were used in your research:
For human MRI, what field strength scanner do you use?
Provide references using APA citation style.
craniotomies in elderly patients in neurosurgery today? A prospective cohort study of 1452
consecutive cases. Journal of Neurosurgery JNS. 2021;134(4):1113–21.
2. Beauregard CL, Friedman WA. Routine use of postoperative ICU care for elective craniotomy:
a cost-benefit analysis. Surg Neurol. 2003;60(6):483–9.
3. Neumann JO, Schmidt S, Nohman A, Naser P, Jakobs M, Unterberg A. Routine ICU surveil-
lance after brain tumor surgery: patient selection using machine learning. J Clin Med.
2024;13(19).
4. Pölsterl S, Wolf TN, Wachinger C. Combining 3D image and tabular data via the dynamic affine
feature map transform. Lecture Notes in Computer Science. Springer International Publishing,
2021:688–98.
5. Wald T, Ulrich C, Lukyanenko S, Goncharov A, Paderno A, Maerkisch L et al. Revisiting
MAE pre-training for 3D medical image segmentation. arXiv preprint arXiv:2410.23132.
2024. Currently under review at CVPR 2024.