Poster No:
1859
Submission Type:
Abstract Submission
Authors:
Thomas Close1, Dorota Jarecka2, Christopher Markiewicz3, Yibei Chen2, Satra Ghosh2
Institutions:
1The University of Sydney, Sydney, New South Wales, 2Massachusetts Institute of Technology, Cambridge, MA, 3Stanford University, Stanford, CA
First Author:
Thomas Close
The University of Sydney
Sydney, New South Wales
Co-Author(s):
Yibei Chen
Massachusetts Institute of Technology
Cambridge, MA
Satra Ghosh
Massachusetts Institute of Technology
Cambridge, MA
Introduction:
Data from neuroimaging experiments require complex analyses to interpret. These analyses often utilize algorithms from various software packages, and consistency and reproducibility demand they be organized into automated pipelines. Nipype has been helping scientists build workflows for almost a decade, providing a uniform interface to neuroimaging packages such as FSL, ANTs, AFNI, FreeSurfer, SPM, and many others. This flexibility has made it an ideal basis for popular preprocessing tools, such as fMRIPrep and C-PAC, and several in the Nipreps framework. Here, we present Nipype 2.0, a new ecosystem built to focus on reproducibility, scalability, and modularity.
Methods:
We focused on the design of Nipype to understand the shortcomings and limitations of using the framework as the ecosystem of tools has evolved to encompass containerized execution, GPU-based AI models and execution, and much larger datasets. Based on this introspection and feedback from the user community, we built a new dataflow engine. We also implemented a solution for automating existing interfaces and workflows. We aimed for this new system to use best practices in Python development and packaging to be more modular, scalable, and easily usable by humans and AI agents.
Results:
We developed a new dataflow engine, Pydra, designed as a general-purpose tool to support any scientific domain. Pydra aims to provide a lightweight dataflow engine for computational graph construction, manipulation, and distributed execution, as well as to ensure the reproducibility of scientific pipelines. In Pydra, a dataflow is represented as a graph, where each node is a Task object. The combination of several key features makes Pydra a customizable and powerful dataflow engine: 1) Composable dataflows: Any node of a dataflow graph can be another dataflow graph (i.e., Workflow), allowing for nested dataflows to arbitrary depths. 2) Flexible semantics for creating nested loops over input. Any Task or Workflow can be run over input parameter sets, and the outputs can be recombined (similar concept to the Map-Reduce model, but applies to nested graphs). 3) A content-addressable global cache (hash values are computed for each graph and each Task). It supports the reuse of previously computed and stored dataflows and tasks. 4) Support for Python functions and external (shell) commands. 5) Native container execution support. Any dataflow or task can be executed in an associated container (via Docker or Singularity), enabling greater consistency and reproducibility.
The new interface packages are built using the pydra-tasks-template, each containing interfaces for a specific neuroimaging tool. To ensure compatibility with Pydra, these interfaces must have a specific structure similar to the interface design used in Nipype. We have autoconverted interfaces for many existing Nipype interfaces, allowing the use of these tools with the Pydra engine.
Conclusions:
The evolution of the Nipype ecosystem will make it easier to maintain and manipulate workflows using language models and to incorporate state-of-the-art deep learning-based approaches into the execution. It will also allow for easier creation of benchmarks and leaderboards for multiverse analyses.
Modeling and Analysis Methods:
Activation (eg. BOLD task-fMRI)
fMRI Connectivity and Network Modeling
Methods Development
Neuroinformatics and Data Sharing:
Workflows 1
Informatics Other 2
Keywords:
Computational Neuroscience
Computing
Data analysis
FUNCTIONAL MRI
Informatics
Machine Learning
Open-Source Software
Workflows
1|2Indicates the priority used for review
By submitting your proposal, you grant permission for the Organization for Human Brain Mapping (OHBM) to distribute your work in any format, including video, audio print and electronic text through OHBM OnDemand, social media channels, the OHBM website, or other electronic publications and media.
I accept
The Open Science Special Interest Group (OSSIG) is introducing a reproducibility challenge for OHBM 2025. This new initiative aims to enhance the reproducibility of scientific results and foster collaborations between labs. Teams will consist of a “source” party and a “reproducing” party, and will be evaluated on the success of their replication, the openness of the source work, and additional deliverables. Click here for more information.
Propose your OHBM abstract(s) as source work for future OHBM meetings by selecting one of the following options:
I do not want to participate in the reproducibility challenge.
Please indicate below if your study was a "resting state" or "task-activation” study.
Other
Healthy subjects only or patients (note that patient studies may also involve healthy subjects):
Healthy subjects
Was this research conducted in the United States?
Yes
Are you Internal Review Board (IRB) certified?
Please note: Failure to have IRB, if applicable will lead to automatic rejection of abstract.
Not applicable
Were any human subjects research approved by the relevant Institutional Review Board or ethics panel?
NOTE: Any human subjects studies without IRB approval will be automatically rejected.
Not applicable
Were any animal research approved by the relevant IACUC or other animal research panel?
NOTE: Any animal studies without IACUC approval will be automatically rejected.
Not applicable
Please indicate which methods were used in your research:
Computational modeling
Which processing packages did you use for your study?
AFNI
SPM
FSL
Free Surfer
Provide references using APA citation style.
not applicable
No