Normative Modeling Workshop: Theory and applications to code along.

Johanna Bayer Organizer
Donders Institute for Brain, Cognition and Behaviour
Nijmegen, Gelderland 
charlotte fraza Co Organizer
Donders Institute for Brain, Cognition and Behaviour
Barbora Rehak Buckova Co Organizer
Radboud UMC
Sunday, Jun 23: 9:00 AM - 1:00 PM
Educational Course - Half Day (4 hours) 
Room: Conference Room E 6 
The normative modelling framework is a new technique that has emerged as an invaluable tool for mapping life-span growth curves in various features derived from neuroimaging, including but notwithstanding structural organization, connectome or complex brain-body relationships. It also allows for an alternative approach for site effect correction in pooled neuroimaging data sets. However, in order to derive and apply these models, novel computational frameworks have to be developed in order to allow appropriate inferences. Consequently, this course aims to introduce a diverse palette of state-of-art normative modelling techniques which cover the latest advances in the field, offering the audience a comprehensive understanding of their advantages and limitations, while also equipping them with the necessary computational resources.
This course will teach attendees to fit, use, interpret and visualize normative models of brain imaging data. The course will further dive into advanced topics of normative modelling, including advanced distributions (SHASH, GAMLSS), longitudinal models and brain imaging data types in a variety of programming languages (R, python).


The overall objective is to provide participants with hands-on experience and tools to be able to transfer knowledge and to use normative modelling on their own data.

- Introduction to Normative Modeling: We aim to acquaint participants with the concept of normative modelling and showcase situations where its application is both appropriate and beneficial.
- Comprehensive Understanding of Methods: We will introduce the most common normative modelling methods, providing practical insights into their implementation. This encompasses a spectrum of approaches, ranging from purely statistical to Bayesian methods, with a focus on highlighting their respective strengths and potential challenges.
- Visualization and interpretation of Results: We will guide participants in understanding and interpreting the results of the models, emphasizing their relevance in subsequent analyses. Special attention will be given to creating visualizations and to illuminating potential limitations that participants should be aware of.
- Advanced theory and applications of normative modelling

We will further illustrate advanced and applied uses of normative modelling. These will include:
- Longitudinal models
- Application to diverse brain imaging measures, such as functional connectivity and in relation with brain/body age
- Advanced modelling approaches, such as models, such as GAMLSS and of the SHASH distribution
- Transfer and prediction of normative modeling results to new and unseen sites and populations 

Target Audience

This course is targeted to researchers who wish to a) understand the theory behind normative modelling b) learn about the use cases and from real live examples of normative modelling application c) learn how to integrate normative modelling into their own research. No previous knowledge of normative modelling is required, and the course is open to all levels of academic seniority. Participants are required to bring their own laptop and expected to follow along/solve small programming tasks.The course does require some basic familiarity with Python, R, Jupyter notebooks, and a GitHub account. 


1. Introduction to normative modelling

10 Minutes introduction and agenda
We will take 10 minutes for introductions.

20 Minutes introduction to normative modeling
Normative modeling is an emerging methodology for comprehending variation within populations. Much like growth charts used in pediatric medicine to chart the distribution of children's height or weight with respect to their age and sex, normative models are key in modeling the distribution of neuroimaging-derived traits within a population. These models primarily focus on establishing the mean and centiles of variation (Marquand et al., 2019) across different parameters like age, gender, or other demographic and clinical variables (Marquand et al., 2016a). Individual deviations from the normative range are captured through deviation or Z-scores, which can then be correlated with various psychiatric disorders.

This tutorial aims to provide students with a first understanding of normative modeling using various Jupyter notebooks. Firstly, participants will be constructing normative models from scratch using open source neuroimaging datasets. Following this, the tutorial will demonstrate the practical usage of pre-existing normative models to assess and compute deviation scores using their own datasets. Lastly, the tutorial will show methods to interpret and visualize the outcomes derived from the normative modelling framework, providing a deeper understanding of the deviation or z-scores per participant.

(Duration: 30min; Presenter: Charlotte Fraza; Format: Jupyter notebooks in follow along format; Programming language: Python; Libraries and Toolboxes: PCNToolkit and various standard python libraries; Degree of interactivity: 100% )

Relevant papers:
Fraza, Charlotte J., et al. ‘Warped Bayesian Linear Regression for Normative Modelling of Big Data’. NeuroImage, vol. 245, Dec. 2021, p. 118715. PubMed,
Marquand, Andre F., Seyed Mostafa Kia, et al. ‘Conceptualizing Mental Disorders as Deviations from Normative Functioning’. Molecular Psychiatry, vol. 24, no. 10, 2019, pp. 1415–24. PubMed Central,
Marquand, Andre F., Iead Rezek, et al. ‘Understanding Heterogeneity in Clinical Cohorts Using Normative Models: Beyond Case-Control Studies’. Biological Psychiatry, vol. 80, no. 7,



charlotte fraza, Donders Institute for Brain, Cognition and Behaviour Nijmegen

2. Normative Models of Adolescent Executive Function Development Across Assessments and Datasets

Theories of human neurobehavioral development suggest goal-directed cognition (executive function) matures from childhood through adolescence, underlying adolescent risk-taking and the emergence of psychopathology. The development and ultimate maturational timing of such executive functions is often used to demarcate the boundaries of adolescence in clinical practice, research, and policy. Investigations with relatively small datasets or narrow subsets of measures have identified general executive function development, but the specific maturational timing and independence of potential executive function subcomponents remain unknown. Multi-assessment and multi-dataset investigations of normative non-linear trajectories of executive function development afford an opportunity to better resolve these issues. This talk and interactive demonstration will highlight opportunities to construct and compare normative developmental trajectories of executive function across assessments and multiple independent datasets, highlighting methodological opportunities to examine independent and aggregate inferences. Using standard packages from the R programming language I will also highlight techniques to identify significant periods of local change and relative periods of stability in nonlinear trajectories that can be leveraged to inform questions regarding demarcating lifespan periods. Finally, I will highlight new avenues to explore person-level variability and deviations from normative trajectories.

(Duration: 40 min; Presenter: Brenden Tervo-Clemmens; Format: Talk and Live Demonstration in RMarkdown in follow-along format; Programming language: R Language; Libraries and Toolboxes: various statistical R libraries; Degree of interactivity: 60%)

Relevant papers:
Tervo-Clemmens, B., Calabro, F. J., Parr, A. C., Fedor, J., Foran, W., & Luna, B. (2023). A canonical trajectory of executive function maturation from adolescence to adulthood. Nature communications, 14(1), 6922. 


Brenden Tervo-Clemmens, University of Minnesota Minneapolis, MN 
United States

3. Making longitudinal inferences using normative models

The advancement of normative models has significantly improved our ability to draw inferences at the individual level when analyzing neuroimaging data. However, it's crucial to note that all existing models have been developed based on cross-sectional data, primarily due to its prevalence and accessibility. While this choice is justifiable, given the current data landscape, the applicability of these models to infer longitudinal dynamics, a trend gaining popularity in the scientific community, raises questions.

This tutorial addresses the current gap by presenting a method tailored for pre-trained normative (cross-sectional) models, introducing what is termed a "z-diff score." This score, rooted in the Warped Bayesian Linear Regression normative model theory, functions as a type of z-score. Its role is to statistically characterize the observed change between two visits as either significant or non-significant [Fraza 2021, Rehák Bučková 2023].

Initially, I will provide a concise overview of the concept and application of pre-trained normative models. I will demonstrate how to download and execute them using sample data on Google Colab. Subsequently, I will explain the concept of the z-diff score, highlighting its underlying assumptions and data prerequisites. Finally, I will showcase the practical computation of the z-diff score and delineate its potential applications in subsequent analyses.

(Duration: 30 min; Presenter: Barbora Rehák Bučková; Jupyter notebooks in follow along format; Programming language: Python; Libraries and Toolboxes: PCNToolkit and various standard python libraries; Degree of interactivity: 100%)

Relevant papers:
Bučková, Barbora Rehák, et al. Using Normative Models Pre-Trained on Cross-Sectional Data to Evaluate Longitudinal Changes in Neuroimaging Data. bioRxiv, 9 June 2023, p. 2023.06.09.544217. bioRxiv,
Fraza, Charlotte J., et al. ‘Warped Bayesian Linear Regression for Normative Modelling of Big Data’. NeuroImage, vol. 245, Dec. 2021, p. 118715. PubMed, 


Barbora Rehak Buckova, Radboud UMC Nijmegen

4. Charting life-course functional connectome using normative modeling

The emergence, development, and aging of the intrinsic connectome architecture enables the dynamic reorganization of functional specialization and integration throughout the lifespan, contributing to continuous changes in human cognition and behavior. Understanding the spatiotemporal growth process of the typical functional connectome is critical for elucidating network-level developmental principles in healthy individuals and for pinpointing periods of heightened vulnerability or potential. The normative modeling framework provides an invaluable tool for mapping life-span growth curves in the human brain connectome. In this educational course, I will show you how to conduct a comprehensive network normative modeling analysis to delineate the nonlinear trajectories of the functional connectome across multiple scales. Firstly, I will highlight the methodological challenges inherent in charting the functional connectome's development over the lifespan, such as establishing templates and parcellations for varying ages, and constructing individualized networks. Then, we will embark on characterizing lifespan changes and rates of change in the overall patterns of the whole-brain functional connectome. With the construction of the first continuous age-related, system-level atlases across the lifespan, I’ll elucidate the distinct growth patterns across brain systems. Following that, we’ll focus on the spatiotemporal principles that govern connectome growth at a finer vertex-level resolution. Finally, I’ll spotlight the potential clinical application of the life-span normative connectome model, by a multiscale characterization to quantify the heterogeneity of participants with autism spectrum disorder and major depressive disorder using individual deviation scores.

(Duration: 40min; Presenter Lianglong Sun; Format: Talk and Live Demonstration in R and Markdown in follow-along format; Programming language: R Language; Libraries and Toolboxes: TBD; Degree of interactivity: 60%)

Relevant papers:
Sun, Lianglong, et al. ‘Functional Connectome through the Human Life Span’. bioRxiv: The Preprint Server for Biology, Sept. 2023, p. 2023.09.12.557193. PubMed,


Lianglong Sun, Beijing Normal University Beijing, Beijing 

5. Methodological advances in normative modelling

Most normative modelling frameworks are limited by specific model assumptions such as gaussianity and homoscedasticity. Further, predictions for new individuals are often limited to sites that have been in the training set of the normative model. This lecture will introduce advanced methodological developments in normative modelling that are able to address these topics, using the PCN-toolkit and extensions in a Jupyter notebook format .
The latest development of normative modelling algorithms include the introduction of the SHASH distribution (Dinga et al.) as the basis of the normative model. Models from the flexible SHASH family allow for the reliable fit of skewed and non-gaussian distributions (Dinga et al., de Boer et al., 2023). In a jupyter notebook and follow-along format using python, attendees will be walked through the settings and deployment of this more complex type of normative model.
The impact of site and scanner effects on the neuroimaging signal have become a highly discussed topic in clinical neuroimaging. In the second half of the session, the focus will be on site information transfer and making predictions for individuals from “unseen sites” (Bayer, et al., 2023). This will be illustrated by walking participants through the site adaptation and prediction step in the jupyter notebook. Further, it will be illustrated how the knowledge on site transfer can be used to access the potential of large, pre-trained normative models (Rutherford et al., 2022).

(Duration: 30min; Presenter: Johanna M. M. Bayer; Format: Jupyter notebooks in follow along format; Programming language: Python; Libraries and Toolboxes: PCNToolkit and various standard python libraries; Degree of interactivity: 100% )

Relevant papers:
de Boer, A. A. A., Kia, S. M., Rutherford, S., Zabihi, M., Fraza, C., Barkema, P., Westlye, L. T.,
Andreassen, O. A., Hinne, M., Beckmann, C. F., & Marquand, A. (2022). Non-Gaussian Normative Modelling With Hierarchical Bayesian Regression. In bioRxiv (p. 2022.10.05.510988).

Dinga, Richard, et al. Normative Modeling of Neuroimaging Data Using Generalized Additive Models of Location Scale and Shape. bioRxiv, 14 June 2021, p. 2021.06.14.448106. bioRxiv,

Bayer, J. M. M., Dinga, R., Kia, S. M., Kottaram, A. R., Wolfers, T., Lv, J., Zalesky, A., Schmaal, L., & Marquand, A. (2022). Accommodating site variation in neuroimaging data using normative and hierarchical Bayesian models. NeuroImage, 264, 119699.

Rutherford, S., Fraza, C., Dinga, R., Kia, S. M., Wolfers, T., Zabihi, M., Berthet, P., Worker, A., Verdi, S., Andrews, D., Han, L. K. M., Bayer, J. M. M., Dazzan, P., McGuire, P., Mocking, R. T., Schene, A., Sripada, C., Tso, I. F., Duval, E. R., … Marquand, A. F. (2022). Charting brain growth and aging at high spatial precision. eLife, 11, e72904. 


Johanna Bayer, Donders Institute for Brain, Cognition and Behaviour Nijmegen, Gelderland 

6. Characterizing brain-body relationships using normative modeling

Integrated research into brain and body systems holds substantial clinical potential in addressing multimorbidity and physical illness burden in people with neuropsychiatric disorders. However, different omics and brain imaging data can have substantially different distributions across individuals. This poses challenges in integrating and comparison of data at multiple levels. Normative modeling provides an excellent opportunity to address this challenge because each biomarker is benchmarked against its own distribution of population norms. Individual variation for a given biomarker is thus standardized and comparable across multiple biophysical and physiological domains. In this educational workshop, I will showcase how we can use normative modeling approaches to characterize brain-body relationships. I will first showcase how to establish normative models of biological age for three-brain and seven-body systems. By doing so, I identified a multiorgan aging network to demonstrate how the aging of one organ system selectively and characteristically influences the aging of other brain and body systems. I will then present a multiorgan, system-wide characterization of brain and body health for common neuropsychiatric disorders. I will show that individuals diagnosed with these neuropsychiatric disorders are not only characterized by deviations from normative reference ranges for brain phenotypes but also present considerably poorer physical health across multiple body systems compared to their healthy peers.

(Duration: 40min; Presenter: Ye Ella Tian; Format: Interactive Presentation; Programming language, Toolboxes and Libraries: N/A:; Degree of interactivity: 20%)

Relevant papers:
Tian, Ye Ella, et al. ‘Evaluation of Brain-Body Health in Individuals With Common Neuropsychiatric Disorders’. JAMA Psychiatry, Apr. 2023. (Crossref),



Ye Tian, University of Melbourne Carlton South, Victoria