Using Nipoppy with Docker

This guide describes the steps required to run Nipoppy with the Docker container engine, for which support was added in version 0.4.2.

Study-level configuration

The following changes need to be made to the "CONTAINER_CONFIG" field of the global_config.json file:

  • "COMMAND" should be set to "docker" instead of "apptainer"

  • Any non-Docker argument/option should be removed from "ARGS"

    • In particular, the default global_config.json uses the "--cleanenv" argument, which is specific to Apptainer and needs to be removed if using Docker.

Tip

To avoid having to manually make these changes for every new Nipoppy study, you can set a custom configuration file at ~/.nipoppy/config.json to be used instead of the default one below. See here for more information.

 1// Nipoppy will apply string substitutions when this file is loaded,
 2// see https://nipoppy.readthedocs.io/en/0.4.8.dev13+g68dba714c/reference/substitutions.html
 3{
 4    "SUBSTITUTIONS": {
 5        // Self-references like NIPOPPY_DPATH_CONTAINERS are resolved from the layout at runtime, making them layout-aware
 6        "[[NIPOPPY_DPATH_CONTAINERS]]": "[[NIPOPPY_DPATH_CONTAINERS]]",
 7        "[[HPC_ACCOUNT_NAME]]": ""
 8    },
 9    "DICOM_DIR_PARTICIPANT_FIRST": true,
10    "CONTAINER_CONFIG": {
11        "COMMAND": "apptainer",
12        "ARGS": [
13            "--cleanenv"
14        ],
15        "BIND_PATHS": [],
16        "ENV_VARS": {
17            "PYTHONUNBUFFERED": "1"
18        }
19    },
20    "HPC_PREAMBLE": [
21        // (These lines can all be removed if not using HPC functionality.)
22        // ========== Activate Python environment ==========
23        // Here we need the command to activate your Python environment in an
24        // HPC job, for example:
25        // - venv:  source <PATH_TO_VENV>/bin/activate
26        // - conda: source ~/.bashrc; conda activate <ENV_NAME>
27        // ========== Set environment variables ==========
28        "export PYTHONUNBUFFERED=1"
29    ],
30    "HPC_QUEUE_LIMIT": 1000,
31    "PIPELINE_VARIABLES": {
32        "BIDSIFICATION": {},
33        "PROCESSING": {},
34        "EXTRACTION": {}
35    },
36    "CUSTOM": {},
37    "SCHEMA_VERSION": "1.0"
38}

Attention

The "BIND_PATHS" field was only added in version 0.4.6. For older versions of Nipoppy, use the "ARGS" with the Docker-specific bind flag --volume, and make sure to specify explicitly the target (destination) path (e.g. --volume /source:/target instead of --volume /source).

Pipeline-level configuration

Similarly to global_config.json, pipeline-specific config.json files also have "CONTAINER_CONFIG" fields. They can appear at the top level or inside individual step configurations. The same changes described above need to be applied to these "CONTAINER_CONFIG" fields.

 1{
 2    "NAME": "fmriprep",
 3    "VERSION": "24.1.1",
 4    "CONTAINER_INFO": {
 5        "FILE": "[[NIPOPPY_DPATH_CONTAINERS]]/[[PIPELINE_NAME]]_[[PIPELINE_VERSION]].sif",
 6        "URI": "docker://nipreps/[[PIPELINE_NAME]]:[[PIPELINE_VERSION]]"
 7    },
 8    "CONTAINER_CONFIG": {
 9        "ENV_VARS": {
10            "TEMPLATEFLOW_HOME": "[[TEMPLATEFLOW_HOME]]"
11        },
12        "BIND_PATHS": [
13            "[[FREESURFER_LICENSE_FILE]]",
14            "[[TEMPLATEFLOW_HOME]]",
15            "[[NIPOPPY_DPATH_PIPELINE_WORK]]:/work",
16            "[[NIPOPPY_DPATH_DERIVATIVES]]/freesurfer/7.3.2/output/[[NIPOPPY_BIDS_SESSION_ID]]"
17        ]
18    },
19    "STEPS": [
20        {
21            "INVOCATION_FILE": "invocation.json",
22            "DESCRIPTOR_FILE": "descriptor.json",
23            "HPC_CONFIG_FILE": "hpc.json",
24            "TRACKER_CONFIG_FILE": "tracker.json"
25        }
26    ],
27    "VARIABLES": {
28        "FREESURFER_LICENSE_FILE": "Path to FreeSurfer license file",
29        "TEMPLATEFLOW_HOME": "Path to the directory where TemplateFlow will store templates (can be empty)"
30    },
31    "PIPELINE_TYPE": "processing",
32    "SCHEMA_VERSION": "1.0"
33}