String substitutions

Nipoppy allows string substitutions when loading configuration files. These serve two purposes:

  • Dynamically resolving values that are unknown prior to runtime (e.g., directory path, pipeline name, participant ID)

  • Avoiding duplication and manual user errors from copy-pasting

User-defined substitutions

Custom substitutions can be defined via the SUBSTITUTIONS field in the global configuration file. They are applied to the global configuration file itself and are propagated to pipeline configuration files.

Attention

User-defined substitutions are applied before the predefined substitutions listed below.

Predefined substitutions

Common substitutions

These substitutions are available for all configuration files:

  • [[NIPOPPY_DPATH_ROOT]]: path to the root directory of the Nipoppy study

  • [[NIPOPPY_DPATH_BIDS]]: path to the directory containing raw BIDS data

  • [[NIPOPPY_DPATH_DERIVATIVES]]: path to the parent derivatives directory

  • [[NIPOPPY_DPATH_PRE_REORG]]: path to the directory containing imaging data before reorganization

  • [[NIPOPPY_DPATH_POST_REORG]]: path to the directory containing imaging data after reorganization

  • [[NIPOPPY_DPATH_CODE]]: path to the code directory

  • [[NIPOPPY_DPATH_PIPELINES]]: path to the directory containing pipeline configurations

  • [[NIPOPPY_DPATH_CONTAINERS]]: path to the directory storing container images

  • [[NIPOPPY_DPATH_LOGS]]: path to the logs directory

  • [[NIPOPPY_DPATH_TABULAR]]: path to the directory container curated tabular phenotypic data

  • [[NIPOPPY_FPATH_CONFIG]]: path to the global configuration file

  • [[NIPOPPY_FPATH_MANIFEST]]: path to the manifest file

  • [[NIPOPPY_FPATH_CURATION_STATUS]]: path to the curation status file

  • [[NIPOPPY_FPATH_PROCESSING_STATUS]]: path to the processing status file

  • And other strings of form [[NIPOPPY_<LAYOUT_PROPERTY>]], where <LAYOUT_PROPERTY> is a property in the Nipoppy dataset layout configuration file (all uppercase)

Pipeline configuration files

These configuration files also have additional utility substitutions (note that they are not prefixed with NIPOPPY_):

  • [[PIPELINE_NAME]]: name of the pipeline

  • [[PIPELINE_VERSION]]: version of the pipeline

  • [[STEP_NAME]]: name of the pipeline step (only within pipeline step configurations)

BIDSification pipeline invocation files

Additional substitutions:

  • [[NIPOPPY_PARTICIPANT_ID]]: the participant ID without the sub- prefix

  • [[NIPOPPY_SESSION_ID]]: the session ID without the ses- prefix

  • [[NIPOPPY_BIDS_PARTICIPANT_ID]]: the participant ID with the sub- prefix

  • [[NIPOPPY_BIDS_SESSION_ID]]: the session ID with the ses- prefix

Processing pipeline invocation files

Additional substitutions:

  • [[NIPOPPY_DPATH_PIPELINE_OUTPUT]]: the output directory for this pipeline, i.e. <NIPOPPY_PROJECT_ROOT>/derivatives/<PIPELINE_NAME>/<PIPELINE_VERSION>/output

  • [[NIPOPPY_DPATH_PIPELINE_WORK]]: the working directory for this pipeline run, which will be a subdirectory of <NIPOPPY_PROJECT_ROOT>/scratch/work/<PIPELINE_NAME>-<PIPELINE_VERSION>/<PIPELINE_NAME>-<PIPELINE_VERSION>

  • [[NIPOPPY_PARTICIPANT_ID]]: the participant ID without the sub- prefix

  • [[NIPOPPY_SESSION_ID]]: the session ID without the ses- prefix

  • [[NIPOPPY_BIDS_PARTICIPANT_ID]]: the participant ID with the sub- prefix

  • [[NIPOPPY_BIDS_SESSION_ID]]: the session ID with the ses- prefix

Extraction pipeline invocation files

Additional substitutions:

  • [[NIPOPPY_DPATH_PIPELINE_IDP]]: the IDP directory for this pipeline, i.e. <NIPOPPY_PROJECT_ROOT>/derivatives/<PIPELINE_NAME>/<PIPELINE_VERSION>/idp

  • [[NIPOPPY_DPATH_PIPELINE_WORK]]: the working directory for this pipeline run, which will be a subdirectory of <NIPOPPY_PROJECT_ROOT>/scratch/work/<PIPELINE_NAME>-<PIPELINE_VERSION>/<PIPELINE_NAME>-<PIPELINE_VERSION>

  • [[NIPOPPY_PARTICIPANT_ID]]: the participant ID without the sub- prefix

  • [[NIPOPPY_SESSION_ID]]: the session ID without the ses- prefix

  • [[NIPOPPY_BIDS_PARTICIPANT_ID]]: the participant ID with the sub- prefix

  • [[NIPOPPY_BIDS_SESSION_ID]]: the session ID with the ses- prefix

Tracker configuration files

Additional substitutions:

  • [[NIPOPPY_BIDS_PARTICIPANT_ID]]: the participant ID with the sub- prefix

  • [[NIPOPPY_BIDS_SESSION_ID]]: the session ID with the ses- prefix

  • [[NIPOPPY_DPATH_PIPELINE_OUTPUT]]: the output directory for this pipeline, i.e. <NIPOPPY_PROJECT_ROOT>/derivatives/<PIPELINE_NAME>/<PIPELINE_VERSION>/output

  • [[NIPOPPY_DPATH_PIPELINE_IDP]]: the IDP directory for this pipeline, i.e. <NIPOPPY_PROJECT_ROOT>/derivatives/<PIPELINE_NAME>/<PIPELINE_VERSION>/idp

  • [[NIPOPPY_PARTICIPANT_ID]]: the participant ID without the sub- prefix

  • [[NIPOPPY_SESSION_ID]]: the session ID without the ses- prefix