How to combine many ROIs (+60) into one ROI?

Hi,

I usually use 3dcalc to combine several ROIs into one ROI, such as via the following code:


3dcalc \
-a 1.nii \
-b 2.nii \
-c 3.nii \
-expr 'step(a+b+c)' \
-prefix Network.nii

But how can I combine 60, 100, or even 400 ROIs into one ROI using AFNI? Even if that was possible with 3dcalc, and as I understand it is not, writing 400 lines of code is prone to errors. What would be the best way to handle this job in AFNI?

Update one:
I found a nice solution by Peter Molfese, here: https://blog.cogneurostats.com/2014/04/25/adventures-in-afni-roi-combinations/
I tried this out, meaning the following steps:

  1. 3dcalc: Create a zero dataset using 3dcalc.
  2. 3dTcat: Concatenate the zero dataset with the specific ROIs one wants to combine.
  3. 3dTstat: Assign number to the single ROIs
  4. 3dcalc: Transform the dataset into a byte type.

Even though this works, thanks to the tutorial by Peter Molfese, I am happy to hear general comments or advice. For example, is there an even faster or better way to do this? Or would this be the way to go? Thanks.

Update two:
Another specific question came up. I am not sure if this question is really related to AFNI, or more to shell scripting. But I believe it is related to AFNI, let me explain. I am using z shell (zsh) on a Mac.

Think of 100 .nii ROI files stored in one folder, such as 1.nii, 2.nii, 3.nii, and so on. Now I would like to combine them using 3dTcat, as suggested by Peter Molfese.
Here is the relevant part of the code that works just fine.


# Concatenate zero dataset with rois
3dTcat \
-prefix Temp \
Zero_dataset+tlrc \
$directory_rois/{1..100}.nii

The code above combines ROIs 1 to 100 into a new output, called “Temp”.

Now, imagine that I want to combine ROIs 1 to 100 together with ROIs 150 to 300. It is possible to build arrays in zsh, such as via:


array=({1..100}.nii {150..300}.nii)

I tried include the variable “array” into the AFNI code above as follows:


# Concatenate zero dataset with rois
3dTcat \
-prefix Temp \
Zero_dataset+tlrc \
$directory_rois/$array

That does not work and produces the following error message:


** FATAL ERROR: Can't open dataset 2.nii

I already tried many alternatives to include the variable’s array into 3dTcat, which I won’t include here in order to prevent making this post too messy and complicated, but AFNI always fails with different errors. It always works fine when using only one brace expansion, such as {1…100}. The problem with AFNI appears once I start using two, such as {1…100} {150…300}. The code


echo  {1..100} {101..300}

prints all numbers from 1 to 300 in zsh, meaning it works fine from the perspective of the z shell.

Is there a way to define a specific array as variable (as shown above), and to then include this array into 3dTcat? Phrased differently: can I somehow combine many ROIs together that do not correspond to a continous list?

Hi Philipp,

What are the current values in the ROIs, and what would you like the values in the output to be? The 2 expected cases are either 0/1 or 0/VALUE (for input, say), where VALUE is perhaps an ROI index. Then the output might be either 0/1 or 0,…,MAX_VALUE, where MAX_VALUE would either be the number of ROIs or perhaps even some larger number, where ROI IDs are not necessarily 1,2,3,…

We could probably give good ways to go with these, but for a precise answer, it would help to know what the inputs and outputs are.

Also, this might be easier if you used zero-padded naming, such as 001.nii, 002.nii, …, 010.nii, …, 100.nii. That would enable use of wildcards, rather than having to fight with {} and counting syntax. But still, either should work.

Note that doing this ir problematic, since the first line will make $array already filled with spaces. But you want $directory_rois attached when it expands:

array=({1…100}.nii {150…300}.nii)
3dTcat -prefix Temp Zero_dataset+tlrc $directory_rois/$array

So instead, consider the more direct:

3dTcat -prefix Temp Zero_dataset+tlrc \
   $directory_rois/{1..100}.nii $directory_rois/{150..300}.nii

Alternatively, you can use an $array variable, but it should not be initially evaluated, and having a space in it is problematic. So 2 vars would help:

a1='{1..100}.nii'
a2='{150..300}.nii'
3dTcat -prefix Temp Zero_dataset+tlrc  \
   `eval echo $directory_rois/$a1`     \
   `eval echo $directory_rois/$a2`

Ugly, but it should work.

And lastly, if the filename were zero-padded in the first place, it might be as simple as:

3dTcat -prefix Temp Zero_dataset+tlrc $directory_rois/*.nii

But to be sure, it would be nice to know about the values in the input and output.

  • rick

Hi,

thank you for your scripting suggestions, much appreciated.
Let me provide some further background to answer your question.

The input ROIs stem from the 1000 parcellation Schaefer-Yeo 17 networks recently updated by you guys:
https://afni.nimh.nih.gov/pub/dist/HBM2021/Schaefer-Yeo_AFNI_Atlas_OHBM2021_Poster.pdf
https://afni.nimh.nih.gov/pub/dist/atlases/SchaeferYeo/

My aim was to use the 1000 parcellations, and then to individually combine the single ROIs/parcellations into the 17 networks. There are a couple of textfiles in one of the folders from the second link above. For example, the textfile “atlaslabels_17N_1000.txt” lists all 1000 ROIs/parcellations and shows to which of the 17 networks they belong.

This is the whole rationale for what I am trying to do here, and in fact, it worked now. I solved the problem as follows. Within my script, I defined conditionals using if.


if [[ $network == VisCent ]] then
array1=1..34
array2=501..539
elif [[ $network == VisPeri ]] then
array1=35..66
array2=540..568
and so on...

Then, I ran a for loop, including all 17 network names. In this four loop, the relevant part for 3dTcat looks as follows:


# Concatenate zero dataset with rois
3dTcat \
-prefix Temp.nii \
Zero_dataset.nii \
$directory_rois/{{$array1}.nii,{$array2}.nii}

This worked well. I skipped the last step (step 4) suggested by P. Molfese. More precisely, I did not run 3dcalc again in the end. All I now did was:

  1. 3dcalc: Create a zero dataset using 3dcalc.
  2. 3dTcat: Concatenate the zero dataset with the specific ROIs one wants to combine.
  3. 3dTstat: Assign number to the single ROIs

Concerning your question: the output masks or ROIs show different values: 0 for black areas (background), different numbers (such as 1, 2, 3 or 20, 21, 22) depending on the single ROIs that were combined to create one network.

As far as my understanding goes, that should be fine, correct? Hence, the step number 4 (using 3dcalc to transform the 17 network .nii files that I created back to a byte type is not really required, or is it)?.

By the way:
I am using


{{$array1}.nii,{$array2}.nii}

because I wanted to combine the left and right hemisphere parcellations together into one network. However, the parcellations start with the left hemisphere for all regions, and then the list continues with the right hemisphere regions.

This is the reason why a continous code, such as {1…50}, does not work. For example, I have to combine the regions of the left hemisphere {1…50} with the ones of the right hemisphere, say {90…130}, from the same network. Does that make sense? This lead to the discussed problem above. And this is everything I basically wanted: (1) creating the 17 networks out of the 1000 parcellations; (2) combing the left and right hemisphere rois, so that overall, only 17 networks (ROIs) remain.

Looks like you have figured all this out. Just to add a few tricks to your quiver:

  1. 3dMean. this is a great program for combining lots of data with wildcards on the command line. Each dataset is loaded one at a time, so you are less likely to have memory issues. It doesn’t provide as many statistical functions as 3dTstat though.

3dMean -prefix testschaef.nii.gz -mask_union ~/afni_sy_atlas/Schaefer_17N_400.nii.gz’<1…200>’ … # more datasets on the command line

  1. Globbing. AFNI commands from compiled C code can glob (expand filenames from wildcard characters) instead of the shell. This is a relatively new addition to AFNI that’s described in the help for afni_open. It does the wildcard expansion in the C code and avoids line length limitations of the shell.

https://afni.nimh.nih.gov/pub/dist/doc/program_help/afni_open.html

3dMean -DAFNI_GLOB_SELECTORS=YES -prefix testschaefmulti.nii.gz -mask_union
‘<<gm /Some/long/path/afni_sy_atlas/Schaefer_17N_?00.nii.gz’“<${rangesel}>”

That doesn’t have a different range for each dataset, which is what you want, I think, but still generally useful. 3dinfo can show the history of how the command expanded with the globbing. Also see gen_group_command.py for some similar functionality.

  1. Concatenated datasets on the command line. AFNI C programs can take lists of datasets as a single concated dataset as one way to “remaster” a dataset. This is particularly useful with programs that take a single dataset input that’s usually across time, like 3dTstat. The shell is doing the globbing here, but you can still see the result with 3dinfo on the output.

export dsetlist=(/Users/glend/afni_sy_atlas/Schaefer_17N_?00.nii.gz)
3dTstat -nzcount -prefix test3dts.nii.gz -mrange 1 200 “$dsetlist”

All these methods combine data without first extracting the datasets to a separate file, so that can reduce disk access.

BTW, Pete’s method converts to byte because 3dfractionize only takes byte as input. That’s a particular way of handling affine transformations that’s useful for moving ROIs, so that’s not relevant to your scripting here. Byte mode also helps cut down on memory usage by half vs the 16-bit short integer. Probably not critical here either.