Improving Braincharts’ calibration reliability by utilising longitudinal information
Poster No:
1552
Submission Type:
Abstract Submission
Authors:
Christopher Adamson1, Chris Moran2, Amy Brown3, Taya Collyer4, Stacey Sakowski5, Valendai Srikanth4, Elisabeth Northam3, Eva Feldman5, Fergus Cameron3, Richard Beare6
Institutions:
1Monash University, Clayton, Victoria, 2School of Public Health and Preventive Medicine, Monash University, Melbourne, Victoria, 3Murdoch Childrens Research Institute, Parkville, Victoria, 4National Centre for Healthy Ageing, Frankston, Victoria, 5Department of Neurology, University of Michigan, Ann Arbor, MI, 6Monash University, Melbourne, Victoria
First Author:
Co-Author(s):
Introduction:
Longitudinal brain imaging studies offer valuable insight into trajectories of brain structures across age. Changes in acquisition hardware can occur between waves introducing systemic biases in resulting measures such as cortical thickness (Han et al., 2006). Calibration methods are required to separate biological and scanner effects.
Braincharts provides a framework for such a calibration method for Freesurfer-derived brain structure measurements (Bethlehem et al., 2022). Firstly, Braincharts contains GAMLSS-based mixed effects models that predict distribution parameters of brain measurements from subject characteristics. Per-site random effects are used to correct for biases. Braincharts' calibration method involves estimating site effects that optimally transform the Brainchart distributions to those of the control subjects in the novel study.
Sites with small subject numbers (≤100) are known to give unstable site effect parameter estimates. Estimation of out-of-sample site effects are purely cross-sectional. Under the assumption that centiles of control subjects are stable across sites, we present a method that to improve reliability of site effect parameter estimation for timepoints in longitudinal studies that have small numbers of control participants but repeat scans of some control participants at multiple timepoints.
Braincharts provides a framework for such a calibration method for Freesurfer-derived brain structure measurements (Bethlehem et al., 2022). Firstly, Braincharts contains GAMLSS-based mixed effects models that predict distribution parameters of brain measurements from subject characteristics. Per-site random effects are used to correct for biases. Braincharts' calibration method involves estimating site effects that optimally transform the Brainchart distributions to those of the control subjects in the novel study.
Sites with small subject numbers (≤100) are known to give unstable site effect parameter estimates. Estimation of out-of-sample site effects are purely cross-sectional. Under the assumption that centiles of control subjects are stable across sites, we present a method that to improve reliability of site effect parameter estimation for timepoints in longitudinal studies that have small numbers of control participants but repeat scans of some control participants at multiple timepoints.
Methods:
For a brain outcome measure y_i from a collection of healthy training subjects i, braincharts provides a model-derived distribution as follows:
y_i∼Ga(μ ̂_i,σ ̂_i,ν ̂_i)
Where Ga denotes the Generalised Gamma Distribution with location parameter μ ̂_i and shape parameters σ ̂_i,ν ̂_i. Calibration of a new site k containing healthy subjects' indexed by j, and brain outcome measurements y=y_jk involves estimating random effects are maximum likelihood estimator that maximizes the following expression:
logp(μ ̂_k,σ ̂_k |y)=∑_k▒[⏟(logp(μ ̂_k )+logp(σ ̂_k ) )┬priors +⏟(∑_j▒〖 log〗l(y_jk ) )┬likelihood ]
y_jk∼Ga(μ ̂_j+μ ̂_k,σ ̂_j+σ ̂_k,ν ̂_i)
μ ̂_k∼N(0,σ_μ )
σ ̂_k∼N(0,σ_σ )
N is the normal distribution. σ_μ,σ_σ denote variances of site effects in the model.
Under the assumption that centiles of control subjects should remain stable across timepoints, we propose a penalty term to penalise discrepancies in centiles as follows:
logp(μ ̂_k,σ ̂_k |y)=∑_k▒[⏟(logp(μ ̂_k )+logp(σ ̂_k ) )┬priors +⏟(∑_j▒〖 log〗l(y_jk ) )┬likelihood ] +∑_k▒∑_j▒|q_0j-q_kj |
Where q_kj denotes the centile of the measurement for novel site k and subject j.
We evaluate the performance of the proposed method using cross-validation on simulated data according to the following procedure. Simulate 1500 subjects with repeated measures at two "sites", a "small" site and a "large" site. Each cross-validation fold is comprised of 5 and 500 single-measurements from the "small" and "large" sites, respectively, plus n_shared repeated measures. The parameters of interest are the site effects for the "small" site with varying numbers of longitudinal subjects. We assess the stability of the estimated μ ̂_k,σ ̂_k for the smaller site across a range of n_shared values that create "small" sites of 10 to 200 subjects. The ground truth parameters μ_k^*,σ_k^* for the small site are those estimated from the full 1500 subjects using the original cross-sectional approach.
y_i∼Ga(μ ̂_i,σ ̂_i,ν ̂_i)
Where Ga denotes the Generalised Gamma Distribution with location parameter μ ̂_i and shape parameters σ ̂_i,ν ̂_i. Calibration of a new site k containing healthy subjects' indexed by j, and brain outcome measurements y=y_jk involves estimating random effects are maximum likelihood estimator that maximizes the following expression:
logp(μ ̂_k,σ ̂_k |y)=∑_k▒[⏟(logp(μ ̂_k )+logp(σ ̂_k ) )┬priors +⏟(∑_j▒〖 log〗l(y_jk ) )┬likelihood ]
y_jk∼Ga(μ ̂_j+μ ̂_k,σ ̂_j+σ ̂_k,ν ̂_i)
μ ̂_k∼N(0,σ_μ )
σ ̂_k∼N(0,σ_σ )
N is the normal distribution. σ_μ,σ_σ denote variances of site effects in the model.
Under the assumption that centiles of control subjects should remain stable across timepoints, we propose a penalty term to penalise discrepancies in centiles as follows:
logp(μ ̂_k,σ ̂_k |y)=∑_k▒[⏟(logp(μ ̂_k )+logp(σ ̂_k ) )┬priors +⏟(∑_j▒〖 log〗l(y_jk ) )┬likelihood ] +∑_k▒∑_j▒|q_0j-q_kj |
Where q_kj denotes the centile of the measurement for novel site k and subject j.
We evaluate the performance of the proposed method using cross-validation on simulated data according to the following procedure. Simulate 1500 subjects with repeated measures at two "sites", a "small" site and a "large" site. Each cross-validation fold is comprised of 5 and 500 single-measurements from the "small" and "large" sites, respectively, plus n_shared repeated measures. The parameters of interest are the site effects for the "small" site with varying numbers of longitudinal subjects. We assess the stability of the estimated μ ̂_k,σ ̂_k for the smaller site across a range of n_shared values that create "small" sites of 10 to 200 subjects. The ground truth parameters μ_k^*,σ_k^* for the small site are those estimated from the full 1500 subjects using the original cross-sectional approach.
Results:
Figures 1 and 2 show the confidence intervals for the small site μ ̂_k and σ ̂_kestimates for values of n_shared. Figure 1 shows comparable and improved confidence intervals for n_shared50, respectively. Figure 2 shows that the proposed method converges quicker than the cross-sectional method and the confidence intervals always overlap the ground truth.
·Confidence intervals for μ ̂_k the small site using the proposed and original methods, respectively.
·Confidence intervals for σ ̂_k the small site using the proposed and original methods, respectively.
Conclusions:
The BrainCharts calibration procedure is unstable when applied to small cohorts acquired on out-of-sample scanners. Our novel, extended procedure supports study designs with repeated measures and stabilizes statistical findings, thus maximising the value of MRI data in established cohorts.
Modeling and Analysis Methods:
Classification and Predictive Modeling 2
Methods Development 1
Keywords:
Cortex
Morphometrics
Other - multisite calibration
1|2Indicates the priority used for review
Abstract Information
By submitting your proposal, you grant permission for the Organization for Human Brain Mapping (OHBM) to distribute your work in any format, including video, audio print and electronic text through OHBM OnDemand, social media channels, the OHBM website, or other electronic publications and media.
The Open Science Special Interest Group (OSSIG) is introducing a reproducibility challenge for OHBM 2025. This new initiative aims to enhance the reproducibility of scientific results and foster collaborations between labs. Teams will consist of a “source” party and a “reproducing” party, and will be evaluated on the success of their replication, the openness of the source work, and additional deliverables. Click here for more information. Propose your OHBM abstract(s) as source work for future OHBM meetings by selecting one of the following options:
Please indicate below if your study was a "resting state" or "task-activation” study.
Healthy subjects only or patients (note that patient studies may also involve healthy subjects):
Was this research conducted in the United States?
Were any human subjects research approved by the relevant Institutional Review Board or ethics panel? NOTE: Any human subjects studies without IRB approval will be automatically rejected.
Were any animal research approved by the relevant IACUC or other animal research panel? NOTE: Any animal studies without IACUC approval will be automatically rejected.
Please indicate which methods were used in your research:
Which processing packages did you use for your study?
Provide references using APA citation style.
Han, X., Jovicich, J., Salat, D., van der Kouwe, A., Quinn, B., Czanner, S., Busa, E., Pacheco, J., Albert, M., Killiany, R., Maguire, P., Rosas, D., Makris, N., Dale, A., Dickerson, B., & Fischl, B. (2006). Reliability of MRI-derived measurements of human cerebral cortical thickness: The effects of field strength, scanner upgrade and manufacturer. NeuroImage, 32(1), 180–194. https://doi.org/10.1016/j.neuroimage.2006.02.051