Creating a script to execute a command for multiple participants simultaneously

AFNI Message Board
1

Hi. I’m new to AFNI and scripting, so forgive me if my query seems rather elementary. I am attempting to do a resting-state analysis and I’d like to execute some commands for each participant simultaneously, so as to avoid having to do them one by one which would be an enormous time investment (and tedious to boot). I have a dataset of masked peak ROIs for each participant and I am using this command to extract the time series from the mask peak for a particular ROI:

3dROIstats -1DRformat -quiet -mask file name > … file.1D

My question is how do I execute this command for multiple participants simultaneously in AFNI, all of which lie in different folders but share the same parent folder, so that I get a 1D file for each participant? Also, I have a number of different ROIs (and masks) which I need to do this for, so can someone please suggest an efficient script which executes this command for certain participants and for a certain ROI?

After this, I will be correlating the 1D file extracted time series with every other voxel using ‘3dTcorr1D’. Again, I would like to do this for a certain set of participants simultaneously. Subsequently, I’ll be converting this file into fisher’s z and talaraich which I would again like to do simultaneously for multiple, select participants. Can someone please demonstrate how I can accomplish this? I’d really appreciate your help!

Thank you in advance,
Jen

2

What you need to do is called “shell scripting” or “shell programming”. An introduction for scripting with the C shell (the one we AFNI-oids prefer) is here:
http://www-cs.canisius.edu/ONLINESTUFF/UNIX/shellprogramming.html

For your particular problem as posed, one solution involves creating and using shell variables, in a loop.

For example


set flist = ( `cat subject.list.txt` )
foreach sub ( $flist )
  cd $sub
  3dROIstats -1DRformat -quiet -mask maskROI.nii stats.nii > ../${sub}.ROI.1D
  cd ..
end

Here, the first shell variable set is the array “flist” which contains the contents of the file “subject.list.txt” – one string per entry in the array. This would be a file something like this:

FTX
FGY
FZN
FLQ
et cetera

where the code name for each subject is on a separate line.
For each subject, the script above assumes there is a directory with that name, and in that directory a dataset named “maskROI.nii” and one named “stats.nii”.

The “foreach” command tells the shell to execute each of the commands, up to the “end”, setting the shell variable “sub” to each of the entries in “flist” in turn. The “$” character is used to substitute the value of a shell variable onto the command line. Thus “cd $sub” will change directory into FTX the first time through the loop, to FGY the second time, and so on. Then the 3dROIstats command is executed in that directory, putting its output into the top level directory with a filename incorporating the subject identifier. Then the script changes directory back up one level and continues through the foreach loop until done.

You will have to understand this, and then adapt the script to the filenames and filename system that you actually use. Doing this requires using the general concepts of programming languages: variables, arrays, loops, variable setting, and variable usage.

If you don’t know anything about programming at all, in any language, you will have a hard time. In which case, try the tutorial link above, and try to find help locally – we won’t be able to teach you programming remotely.

3

Thank you for your advice, Bob. Unfortunately, I can’t apply the command you’ve suggested because my masks lie in separate folders (they are not in the subject directories and I can’t really move things around…I am working with a large external hard drive containing a bunch of different people’s data. It was given to me as is and it’s not very well organized.) Furthermore, the masks all have unique names, and there are 4 different mask sizes for about the 12 ROIs I am investigating. I don’t know how to go about creating a script or finding a way to automate doing these commands for multiple subjects, so I’m just doing them individually for now. I am trying to find help locally. Please allow me to give you some idea of how I’m working through this. I’ll demonstrate what I’m doing as I’m executing this command for a single subject, and maybe you can advise me how to be able to apply this to multiple subjects.

The masks are located here:
Masks / Data / individual folders for all subjects (all folders have unique names) / afni files (multiple masks for all ROIs)

In the command terminal, I enter one of the subject folders, like so:
cd …/Data/Subj4/

So, I’m in the subject’s folder now (where the masks are located, not where the subject’s original/clean afni files are. Those are in another directory that I will be accessing, as I demonstrate below.) Every mask for every ROI is in here, but I’m just applying a single, particular mask (e.g. Ir10_t0) to a single, particular ROI (e.g. L_FFA.) So, the command looks like this:

3dROIstats -1DRformat -quiet -mask unique mask file /path to original file within a subject directory > path to original file within a subject directory/**.1D

Again, forgive me if this seems trivial or if I’ve been unclear. Can you suggest a solution? For simplicity’s sake, I’d like to execute this 1D output command, just taking 1 mask (Ir10_t0) and 1 ROI (L_FFA), and run it simultaneously for all subjects in a directory.

4

Hi Jen,

In order to script this, you need to be able to expect
the names of things like ROI datasets, or at worst,
request them from the command line. Reasonable data
organization is basically a requirement for being able
to script things easily.

And since we cannot really see all of your files (and
perhaps we would not want to :), we cannot say exactly
how to proceed.

One option would be to make a list of subject IDs and
corresponding masks. Then your script could walk
through those lists to retrieve the data.

But all of those ROIs and “path to” entries in the
3dROIstats command need to be predictable.

Otherwise, you could even just copy those datasets
to a new location and give them useful names.

  • rick
5

The simplest possible script is to create a file with 1 line per command you want to execute.

3dROIstats -1DRformat -quiet -mask directory1/inputfile1 > outputfile1.1D
3dROIstats -1DRformat -quiet -mask directory2/inputfile2 > outputfile2.1D

and so on – create the first line, copy it, and then edit it into the format you need. If you are good with an editor and if your filenames have some usable pattern, this is usually not very difficult. It can be helped by making a list of all the input files, and then editing that into the script, as in

ls directory*/inputfile* > script.csh

and then edit file script.csh to add the necessary text around the input filenames.

If you do not understand this posting and how to start doing this, then I’m afraid you will have to seek help elsewhere, and perhaps find a course that can teach you scripting.

6

Sorry, I couldn’t log in earlier to respond. Thank you both for your suggestions and advice. I am receiving local help with this at the moment and have worked out the file paths/scripting issue.