Analysis pipelines framework of the Australian Imaging Service
Poster No:
1861
Submission Type:
Abstract Submission
Authors:
Thomas Close1, Arkiev D'Souza1, Ryan Sullivan2, Fernando Calamante1
Institutions:
1Sydney Imaging, The University of Sydney, Sydney, Australia, 2School of Biomedical Engineering, The University of Sydney, Sydney, Australia
First Author:
Co-Author(s):
Introduction:
The Australian Imaging Service (AIS) is a federated platform for the management of biomedical imaging data (Figure 1). In order to provide standardized, reproducible and scalable analysis methods to the AIS community, a pipelines framework has been developed, which provides a suite of pipelines that can be launched via a web UI or triggered on data ingest. The framework streamlines the deployment of analysis methods to container images that can be run by XNAT (Marcus, 2007) or run as stand-alone BIDS apps (Gorgolewski, 2017). The pipelines framework is designed to improve the reproducibility, scalability, and accessibility of biomedical imaging workflows in diverse research contexts.
Methods:
Pipelines are defined as Pydra "tasks" (Ghosh, 2021), which can be shell commands, Python functions or multi-component workflows. Docker images are built from YAML specifications that define the pipeline task and the required software dependencies.
The core components of the deployment framework are:
1. FrameTree (https://github.com/ArcanaFramework/frametree): FrameTree is a package that overlays virtual frames onto the hierarchical data structures to coordinate the application of analysis methods that operate at different levels of the data structure. It abstracts out how the data is stored, enabling modular design that supports different storage systems such as XNAT and BIDS.
2. Pipeline2App (https://github.com/ArcanaFramework/pipeline2app): Pipeline2App is a package automates the generation of containerized pipeline applications from Pydra tasks. It converts YAML specifications into deployable applications compatible with either XNAT or BIDS.
3. GitHub Actions: Continuous integration and deployment workflows implemented via GitHub Actions automate the build and testing process for new pipelines and deploy them to the AIS network via the GitHub container registry.
These tools are integrated to create a streamlined process for developing biomedical imaging pipelines (Figure 2).
The core components of the deployment framework are:
1. FrameTree (https://github.com/ArcanaFramework/frametree): FrameTree is a package that overlays virtual frames onto the hierarchical data structures to coordinate the application of analysis methods that operate at different levels of the data structure. It abstracts out how the data is stored, enabling modular design that supports different storage systems such as XNAT and BIDS.
2. Pipeline2App (https://github.com/ArcanaFramework/pipeline2app): Pipeline2App is a package automates the generation of containerized pipeline applications from Pydra tasks. It converts YAML specifications into deployable applications compatible with either XNAT or BIDS.
3. GitHub Actions: Continuous integration and deployment workflows implemented via GitHub Actions automate the build and testing process for new pipelines and deploy them to the AIS network via the GitHub container registry.
These tools are integrated to create a streamlined process for developing biomedical imaging pipelines (Figure 2).
Results:
Pipelines deployed with the AIS pipelines framework are split into two repository GitHub repositories, a primary one for centrally curated pipelines, https://github.com/Australian-Imaging-Service/pipelines, and a repository for pipelines contributed by the AIS user community, https://github.com/Australian-Imaging-Service/pipelines-community.
Pipelines are added via GitHub pull-requests that are reviewed by the AIS Pipelines Team. Community-contributed pipelines are placed in separate namespaces based on research group (e.g. au.edu.sydney.sydneyimaging) and security audited before they are accepted. Curated pipelines are released on six-month cycle, upgrading any software dependencies to their latest versions and testing the artefacts produced by the pipelines for regressions between releases.
The framework was validated through the development of multiple pipelines, including complete pre-processing workflows for structural and functional MRI data.
Pipelines are added via GitHub pull-requests that are reviewed by the AIS Pipelines Team. Community-contributed pipelines are placed in separate namespaces based on research group (e.g. au.edu.sydney.sydneyimaging) and security audited before they are accepted. Curated pipelines are released on six-month cycle, upgrading any software dependencies to their latest versions and testing the artefacts produced by the pipelines for regressions between releases.
The framework was validated through the development of multiple pipelines, including complete pre-processing workflows for structural and functional MRI data.
Conclusions:
The pipelines framework streamlines the deployment of biomedical imaging pipelines across the AIS. The framework improves the reproducibility, scalability, and accessibility of analysis methods available to users of the AIS, addressing demand for standardized analysis over large multi-site projects. This approach enables researchers to focus on scientific discovery rather than implementation details, accelerating progress in neuroimaging research and clinical applications.
Future work will focus on expanding the library of available pipelines and enhancing the integration with cloud computing platforms. The AIS Pipelines framework provides a model for other research domains seeking to adopt similar automated and standardized approaches to workflow development.
Future work will focus on expanding the library of available pipelines and enhancing the integration with cloud computing platforms. The AIS Pipelines framework provides a model for other research domains seeking to adopt similar automated and standardized approaches to workflow development.
Neuroinformatics and Data Sharing:
Workflows 1
Informatics Other 2
Keywords:
Data analysis
Data Organization
Informatics
Open-Source Software
Workflows
1|2Indicates the priority used for review
Abstract Information
By submitting your proposal, you grant permission for the Organization for Human Brain Mapping (OHBM) to distribute your work in any format, including video, audio print and electronic text through OHBM OnDemand, social media channels, the OHBM website, or other electronic publications and media.
The Open Science Special Interest Group (OSSIG) is introducing a reproducibility challenge for OHBM 2025. This new initiative aims to enhance the reproducibility of scientific results and foster collaborations between labs. Teams will consist of a “source” party and a “reproducing” party, and will be evaluated on the success of their replication, the openness of the source work, and additional deliverables. Click here for more information. Propose your OHBM abstract(s) as source work for future OHBM meetings by selecting one of the following options:
Please indicate below if your study was a "resting state" or "task-activation” study.
Healthy subjects only or patients (note that patient studies may also involve healthy subjects):
Was this research conducted in the United States?
Were any human subjects research approved by the relevant Institutional Review Board or ethics panel? NOTE: Any human subjects studies without IRB approval will be automatically rejected.
Were any animal research approved by the relevant IACUC or other animal research panel? NOTE: Any animal studies without IACUC approval will be automatically rejected.
Please indicate which methods were used in your research:
For human MRI, what field strength scanner do you use?
Which processing packages did you use for your study?
Provide references using APA citation style.
Presented at the Organization for Human Brain Mapping (OHBM) Annual Meeting, 2021. Available at: https://github.com/nipype/pydra
Gorgolewski, K. J., Alfaro-Almagro, F., & Auer, T. (2017). BIDS Apps: Improving ease of use, accessibility, and reproducibility of neuroimaging data analysis methods. PLoS Computational Biology, 13(3), e1005209. https://doi.org/10.1371/journal.pcbi.1005209
Marcus, D. S., Olsen, T. R., Ramaratnam, M., & Buckner, R. L. (2007). The extensible neuroimaging archive toolkit: An informatics platform for managing, exploring, and sharing neuroimaging data. Neuroinformatics, 5(1), 11–34. https://doi.org/10.1385/NI:5:1:11