Optimizing Torch-MISA for Efficient Signal Separation in IVA of fMRI via Definitive Screening Design
Poster No:
1546
Submission Type:
Abstract Submission
Authors:
Orit Yohannes1, Fareya Borhan1, Poorav Rawat1, Aditya Sule1, Calvin McCurdy1, Noah Lewis2, Bradley Baker2, Xinhui Li3, Vince Calhoun4, Rogers Silva2
Institutions:
1Georgia State University, Atlanta, GA, 2TReNDS Center, Atlanta, GA, 3Georgia Institute of Technology, Atlanta, GA, 4GSU/GATech/Emory, Atlanta, GA
First Author:
Co-Author(s):
Introduction:
The independent vector analysis (IVA) framework (Bhinge, 2019), implemented in Torch-MISA (Li, 2023), is key for blind source separation in multi-subject functional neuroimaging (Fig 1.a). Torch-MISA's performance depends on hyperparameters like learning rate and batch size. However, their interaction in large IVA problems---where computational costs scale quadratically with subject count---is poorly understood. Optimizing these settings could improve both accuracy and efficiency.
This study identifies optimal hyperparameters for large IVA tasks, enhancing source separation accuracy and reducing convergence rates to lower computational costs. We use a Definitive Screening Design (DSD) (Jones, 2017) to evaluate hyperparameter effects with minimal experiments. A quadratic model is then built to capture interactions and nonlinearities, predicting settings for optimal accuracy and convergence.
Confirmatory experiments validate the model's predictions based on the multidataset intersymbol interference (MISI) measure, showing significant gains in computational efficiency while retaining signal separation accuracy.
This study identifies optimal hyperparameters for large IVA tasks, enhancing source separation accuracy and reducing convergence rates to lower computational costs. We use a Definitive Screening Design (DSD) (Jones, 2017) to evaluate hyperparameter effects with minimal experiments. A quadratic model is then built to capture interactions and nonlinearities, predicting settings for optimal accuracy and convergence.
Confirmatory experiments validate the model's predictions based on the multidataset intersymbol interference (MISI) measure, showing significant gains in computational efficiency while retaining signal separation accuracy.
Methods:
We simulated ground-truth brain imaging data for 100 subjects to test Torch-MISA in a controlled environment. The goal was to extract true independent components for each subject while maintaining source alignment between subjects. This study focused on identifying and optimizing key hyperparameters that impact model convergence and accuracy.
To optimize training hyperparameters for Torch-MISA in large IVA problems, we used Definitive Screening Design (DSD). Unlike traditional factorial designs, DSD efficiently captures main effects and second-order interactions with fewer experiments. This method allows rapid exploration of high-dimensional parameter spaces while identifying significant hyperparameters and assessing quadratic effects without confounds.
We studied eight hyperparameters (Fig. 1.b), with learning rate, batch size, and patience using logarithmic scales for their experimental levels. Performance was evaluated with three metrics: Epochs, the number of passes through the entire data until convergence (i.e., when the loss flattens out and learning stops); MISI, the multidataset intersymbol interference measure indicating source separation quality; and MxE, a combined measure of separation quality and training efficiency.
A linear regression model was developed to analyze results. Both a full quadratic model (covering all main effects, interactions, and quadratic terms) and a refined model (including only terms with significant uncorrected p-values, α = 0.01) were used to predict Torch-MISA performance. These models systematically identified hyperparameter combinations that minimize MISI, Epochs, and MxE, achieving improved accuracy and efficient convergence.
To optimize training hyperparameters for Torch-MISA in large IVA problems, we used Definitive Screening Design (DSD). Unlike traditional factorial designs, DSD efficiently captures main effects and second-order interactions with fewer experiments. This method allows rapid exploration of high-dimensional parameter spaces while identifying significant hyperparameters and assessing quadratic effects without confounds.
We studied eight hyperparameters (Fig. 1.b), with learning rate, batch size, and patience using logarithmic scales for their experimental levels. Performance was evaluated with three metrics: Epochs, the number of passes through the entire data until convergence (i.e., when the loss flattens out and learning stops); MISI, the multidataset intersymbol interference measure indicating source separation quality; and MxE, a combined measure of separation quality and training efficiency.
A linear regression model was developed to analyze results. Both a full quadratic model (covering all main effects, interactions, and quadratic terms) and a refined model (including only terms with significant uncorrected p-values, α = 0.01) were used to predict Torch-MISA performance. These models systematically identified hyperparameter combinations that minimize MISI, Epochs, and MxE, achieving improved accuracy and efficient convergence.
·Methodology
Results:
Our screening analysis showed that only learning rate, batch size, beta 1, and beta 2 significantly affected the performance metrics (MISI, Epochs, and MxE). By focusing on these key hyperparameters, we uncovered critical relationships impacting model convergence, signal separation quality, and training efficiency.
Figure 2.a presents the optimal settings identified for each model, while Figure 2.b shows training curves for these settings. The curves closely align with predicted accuracies and convergence epochs, validating the models. Additionally, we developed a combined Setting 7, which dynamically switches between optimal settings during training based on a preset schedule. This approach achieved accurate convergence in just 100 epochs.
Figure 2.a presents the optimal settings identified for each model, while Figure 2.b shows training curves for these settings. The curves closely align with predicted accuracies and convergence epochs, validating the models. Additionally, we developed a combined Setting 7, which dynamically switches between optimal settings during training based on a preset schedule. This approach achieved accurate convergence in just 100 epochs.
·Results
Conclusions:
Tuning hyperparameters is crucial for optimizing Torch-MISA's performance in large IVA problems, improving efficiency and accuracy in multi-subject neuroimaging. Using Definitive Screening Design (DSD), we reduced experiments and computational costs while identifying key hyperparameter relationships. Ongoing work extends this approach to larger datasets, validating scalability and robustness with independent test sets.
Modeling and Analysis Methods:
Exploratory Modeling and Artifact Removal
Methods Development 1
Multivariate Approaches 2
Univariate Modeling
Other Methods
Keywords:
Computing
Data analysis
Design and Analysis
Experimental Design
FUNCTIONAL MRI
Machine Learning
Modeling
Multivariate
Statistical Methods
Other - ICA; IVA
1|2Indicates the priority used for review
Abstract Information
By submitting your proposal, you grant permission for the Organization for Human Brain Mapping (OHBM) to distribute your work in any format, including video, audio print and electronic text through OHBM OnDemand, social media channels, the OHBM website, or other electronic publications and media.
The Open Science Special Interest Group (OSSIG) is introducing a reproducibility challenge for OHBM 2025. This new initiative aims to enhance the reproducibility of scientific results and foster collaborations between labs. Teams will consist of a “source” party and a “reproducing” party, and will be evaluated on the success of their replication, the openness of the source work, and additional deliverables. Click here for more information. Propose your OHBM abstract(s) as source work for future OHBM meetings by selecting one of the following options:
Please indicate below if your study was a "resting state" or "task-activation” study.
Healthy subjects only or patients (note that patient studies may also involve healthy subjects):
Was this research conducted in the United States?
Are you Internal Review Board (IRB) certified? Please note: Failure to have IRB, if applicable will lead to automatic rejection of abstract.
Were any human subjects research approved by the relevant Institutional Review Board or ethics panel? NOTE: Any human subjects studies without IRB approval will be automatically rejected.
Were any animal research approved by the relevant IACUC or other animal research panel? NOTE: Any animal studies without IACUC approval will be automatically rejected.
Please indicate which methods were used in your research:
For human MRI, what field strength scanner do you use?
Which processing packages did you use for your study?
Provide references using APA citation style.
2. Jones, B., & Nachtsheim, C. (2017). Effective Design-Based Model Selection for Definitive Screening Designs. Technometrics, 59(3), 319-329.
3. Li, X., Khosravinezhad, D., Calhoun, V. D., & Silva, R. F. (2023). Evaluating trade-offs in IVA of multimodal neuroimaging using cross-platform multidataset independent subspace analysis. In Proceedings of the IEEE International Symposium on Biomedical Imaging (ISBI), Cartagena, Colombia.