Denoising diffusion networks for normative modeling in neurodegeneration
Poster No:
1507
Submission Type:
Abstract Submission
Authors:
Luke Whitbread1, Lyle Palmer1, Mark Jenkinson1
Institutions:
1The University of Adelaide, Adelaide, South Australia
First Author:
Co-Author(s):
Introduction:
Normative modeling captures the distribution of a value (e.g., hippocampal volume) for a population of normal individuals, as a function of a variable of interest (e.g., age) (Rutherford, 2023); and can identify deviations from typical brain patterns, aiding detection and prognosis of neurodegeneration. Traditional approaches, such as Generealised Additive Models for Location, Scale and Shape (GAMLSS) rely on parametric assumptions that limit flexibility and accuracy, especially for complex, multimodal data. In contrast, deep-learning diffusion models (Sohl-Dickstein, 2015; Nichol, 2021) offer a flexible framework for modeling complex data distributions, enabling the discovery of nuanced dependencies, such as those driven by demographic or genetic factors.
Methods:
Data:
We have generated synthetic scalar IDPs for four brain structures (A--D) using an age covariate (50--96 yrs) and two subgroups, creating realistic distributions beyond the simple mathematical functions used by typical statistical methods. For each age, 1000 datapoints were synthesised with evenly sized subgroups. IDP distributions, at each specific age, are either a unimodal distribution (<65 yrs) or bimodal (≥ 65 yrs). The following mean and standard deviation functions of age were used:
μA(x) = -70(x-65).s((x-65)/10) + 20(x-65) + 7000 +/- H(x-65).x(x-65)/5
σA(x) = 5(x-65).s((x-65)/10) + (x-65) + 300
μB(x) = -200(x-15).s((x-73)/8) + 45000 +/- H(x-65).x(x-65)
σB(x) = 25(x-15).s((x-73)/8) + 4500
μC(x) = exp(-0.04(x-73)).7000 + 25000 +/- H(x-65).x(x-65)
σC(x) = 25(x-15).s((x-73)/8) + 4500
μD(x) = exp(0.02(x-50)).7000 + 15000 +/- H(x-65).x(x-65)
σD(x) = exp(0.03(x-75)).7000 + 5000
where x represents age in years, μA ... μD represent mean functions for the four structures, σA ... σD are the standard deviation functions, H(x) is the Heaviside function that is zero if x<0 and 1 otherwise, and s(.) is the sigmoid function. The +/- in the mean functions represent the two subgroups. The mean and standard deviations parameterise standard Gaussian distributions for subgroups in the first three structures, while for the fourth structure, they parameterise skew-normal distributions with shape parameters of 7.
Diffusion model and training:
We estimated normative models using a Denoising Diffusion Probabilistic Model architecture (Ho, 2020), with an 8-layer MLP backbone (256 neurons, PReLU activations, and dropout) and covariate conditioning via concatenation and FiLM (Perez, 2018). We trained with 50 timesteps, 4000 epochs, a linear β-schedule (0.02--0.95), minibatches of 256, and 0.0001 initial learning rate.
We have generated synthetic scalar IDPs for four brain structures (A--D) using an age covariate (50--96 yrs) and two subgroups, creating realistic distributions beyond the simple mathematical functions used by typical statistical methods. For each age, 1000 datapoints were synthesised with evenly sized subgroups. IDP distributions, at each specific age, are either a unimodal distribution (<65 yrs) or bimodal (≥ 65 yrs). The following mean and standard deviation functions of age were used:
μA(x) = -70(x-65).s((x-65)/10) + 20(x-65) + 7000 +/- H(x-65).x(x-65)/5
σA(x) = 5(x-65).s((x-65)/10) + (x-65) + 300
μB(x) = -200(x-15).s((x-73)/8) + 45000 +/- H(x-65).x(x-65)
σB(x) = 25(x-15).s((x-73)/8) + 4500
μC(x) = exp(-0.04(x-73)).7000 + 25000 +/- H(x-65).x(x-65)
σC(x) = 25(x-15).s((x-73)/8) + 4500
μD(x) = exp(0.02(x-50)).7000 + 15000 +/- H(x-65).x(x-65)
σD(x) = exp(0.03(x-75)).7000 + 5000
where x represents age in years, μA ... μD represent mean functions for the four structures, σA ... σD are the standard deviation functions, H(x) is the Heaviside function that is zero if x<0 and 1 otherwise, and s(.) is the sigmoid function. The +/- in the mean functions represent the two subgroups. The mean and standard deviations parameterise standard Gaussian distributions for subgroups in the first three structures, while for the fourth structure, they parameterise skew-normal distributions with shape parameters of 7.
Diffusion model and training:
We estimated normative models using a Denoising Diffusion Probabilistic Model architecture (Ho, 2020), with an 8-layer MLP backbone (256 neurons, PReLU activations, and dropout) and covariate conditioning via concatenation and FiLM (Perez, 2018). We trained with 50 timesteps, 4000 epochs, a linear β-schedule (0.02--0.95), minibatches of 256, and 0.0001 initial learning rate.
Results:
Our method estimated joint distributions for all four structures given age. We compared results to separate per structure GAMLSS models. Centile errors were calculated using absolute difference to the ground-truth centiles at each age. Figure 1 displays errors and centile curves overlaid on the ground-truth curves. These results show that the two methods perform similarly, with both generally having low errors but with some higher errors cases, of comparable magnitude. This is despite our network being trained jointly for all structures. Figure 2 shows centile errors for both methods with reduced dataset size (50% and 25%) to test stability. Again, both methods perform similarly.
Conclusions:
This study demonstrates the success of a diffusion model-based framework for normative modeling, achieving comparable results to GAMLSS, while jointly estimating multiple structures. Noteably, the diffusion model remains robust to reduced dataset sizes and offers greater flexibility, allowing integration of multiple conditioning factors (e.g., age, education, physical activity, etc.) for more powerful and individualised normative centile estimates. As this is the first work on diffusion models for normative modeling, these results are likely to improve with further model optimisation.
Disorders of the Nervous System:
Neurodegenerative/ Late Life (eg. Parkinson’s, Alzheimer’s) 2
Lifespan Development:
Aging
Modeling and Analysis Methods:
Methods Development 1
Keywords:
Aging
Degenerative Disease
Other - Normative Modeling, Denoising Diffusion Networks
1|2Indicates the priority used for review
Abstract Information
By submitting your proposal, you grant permission for the Organization for Human Brain Mapping (OHBM) to distribute your work in any format, including video, audio print and electronic text through OHBM OnDemand, social media channels, the OHBM website, or other electronic publications and media.
The Open Science Special Interest Group (OSSIG) is introducing a reproducibility challenge for OHBM 2025. This new initiative aims to enhance the reproducibility of scientific results and foster collaborations between labs. Teams will consist of a “source” party and a “reproducing” party, and will be evaluated on the success of their replication, the openness of the source work, and additional deliverables. Click here for more information. Propose your OHBM abstract(s) as source work for future OHBM meetings by selecting one of the following options:
Please indicate below if your study was a "resting state" or "task-activation” study.
Healthy subjects only or patients (note that patient studies may also involve healthy subjects):
Was this research conducted in the United States?
Were any human subjects research approved by the relevant Institutional Review Board or ethics panel? NOTE: Any human subjects studies without IRB approval will be automatically rejected.
Were any animal research approved by the relevant IACUC or other animal research panel? NOTE: Any animal studies without IACUC approval will be automatically rejected.
Please indicate which methods were used in your research:
Provide references using APA citation style.
Nichol, A.Q., & Dhariwal, P. (2021). Improved Denoising Diffusion Probabilistic Models. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research, 139:8162-8171. https://proceedings.mlr.press/v139/nichol21a.html
Perez, E., Strub, F., de Vries, H., Dumoulin, V., & Courville, A. (2018). FiLM: Visual Reasoning with a General Conditioning Layer. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1). https://doi.org/10.1609/aaai.v32i1.11671
Rutherford, S., Barkema, P., Tso, I. F., Sripada, C., Beckmann, C. F., Ruhe, H. G., & Marquand, A. F. (2023). Evidence for embracing normative modeling. eLife, 12, e85082. https://doi.org/10.7554/eLife.85082
Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N., & Ganguli, S. (2015). Deep Unsupervised Learning using Nonequilibrium Thermodynamics. Proceedings of the 32nd International Conference on Machine Learning, in Proceedings of Machine Learning Research, 37:2256-2265. https://proceedings.mlr.press/v37/sohl-dickstein15.html