Square brackets and AFNI - How to really use them in Zsh?

AFNI Message Board
1

Dear all,

I am back again with another question. I am using Zsh for my terminal on a MAC. Since I got used to the language of Zsh in my learning process of AFNI, I would like to stick to it, even though I read around here that you, the AFNI people, recommended to use tcsh. In case you want to comment on this, please, go ahead. I am interested in case you would still suggest to switch to bash, tcsh, or whatever (instead of using Zsh). Otherwise I would stick with Zsh for now.

But lets get back to the problem with Zsh on Mac and the usage of square brackets:
Currently I am using 3dTcat a lot and came upon problems with its usage of square brackets […].

Let me provide you an example. Imagine I cut out distinct time windows from a big run. Like in the following of a code below:

directory=/users/philipp/desktop/fmri/dataset/subjects
for subject in subj1_exp1
do
for fMRIruns (fitts.Subject_1+tlrc)
do
for SelfTimeWindows (‘[21-75]’ ‘[199-253]’ ‘[427-481]’ ‘[489-543]’ ‘[549-603]’ ‘[659-713]’ ‘[837-891]’ ‘[893-947]’ ‘[955-1009]’ ‘[1013-1067]’)
do
cd $directory/$subject/Preprocessing
echo “Processing $subject … $SelfTimeWindows …”
3dTcat
-prefix Self$SelfTimeWindows
-session $directory/$subject/TimeWindows
-verb
-tr 1.0
$fMRIruns.BRIK'$SelfTimeWindows'
done
done
done

Now, what will AFNI do? AFNI will nicely save individual files, e.g., the first output fill will be named “Self[21-75]+tlrc”. Of course it creates both a .BRIK and a .HEAD file for each output file.

First problem: I cannot open those files directly via terminal commands. Why not? Either AFNI tells me that the file does not exist, or that it cannot open them. However, if I now start AFNI and open those files via the GUI, AFNI opens the files just fine. I can inspect them, for example the time-series. Everything looks good. I believe that the problem is the filename itself, more precisely the square brackets of the filename.

Second problem: Lets say that I need those previously cut single time windows in a next step to merge them together into one big run again (for example one run with only self trials, and the other run with only non-self trials). This could be done via the following code:

directory=/users/philipp/desktop/fmri/dataset/subjects
for subject in subj1_exp1
do
for SelfTimeWindows (‘Self[21-75]+tlrc’ ‘Self[199-253].+tlrc’ ‘Self[427-481]+tlrc’ ‘Self[489-543]+tlrc’ ‘Self[549-603]+tlrc’ ‘Self[659-713]+tlrc’ ‘Self[837-891]+tlrc’ ‘Self[893-947]+tlrc’ ‘Self[955-1009]+tlrc’ ‘Self[1013-1067]+tlrc’)
do
for NonSelfTimeWindows (‘Non-Self[81-135]+tlrc’ ‘Non-Self[137-191]+tlrc’ ‘Non-Self[257-311]+tlrc’ ‘Non-Self[367-421]+tlrc’ ‘Non-Self[605-659]+tlrc’ ‘Non-Self[717-771]+tlrc’ ‘Non-Self[779-833]+tlrc’ ‘Non-Self[1073-1127]+tlrc’ ‘Non-Self[1127-1179]+tlrc’)
do
cd $directory/$subject/TimeWindows
echo “Processing $subject … $SelfTimeWindows … $NonSelfTimeWindows …”

3dTcat
-prefix SelfTimeWindows
-session $directory/$subject/TimeWindows
-verb
-tr 1.0
$SelfTimeWindows

3dTcat
-prefix NonSelfTimeWindows
-session $directory/$subject/TimeWindows
-verb
-tr 1.0
$NonSelfTimeWindows

done
done
done

AFNI now has the problem with the square brackets (as I assume) once again. It simply cannot find or open the single time window files. No matter how I change the quoation marks ( ’ ’ ) style, I tried everything forward and back, AFNI either tells me that those files do not exist, or that it cannot open them.

I would assume that the solution to this problem is rather easy, as long as my code above is correct. I assume that I only need to add some small adjustments so that AFNI can read and open those files for processing. But I am lost with what adjustment is required.

In the process of fixing this problem, I had the idea to simply save the cut time windows without the square brackets, so that the file name would not contain square brackets. AFNI does just that, no errors occur; but now comes the problem: it saves the whole run instead of cutting out single time windows. So, it creates distinct files, gives them the correct output name, but every single file actually contains the whole run. (This solution did not work out in Zsh, but strangely via BASH it works.)

So, the problem appears really to be related to Zsh on a MAC and square brackets. It would be nice to not have to switch the code and language for single processing steps in AFNI.

In case you know a solution to this, please let me know.

Philipp

2

An update:

I think that my code to merge the distinct time windows (of the self and non-self runs) is a disaster. With the code displayed in my first post, AFNI would try to do the following:

  1. It would assume that it has to take each time window in the two for loops (for SelfTimeWindows & for NonSelfTimeWindows) and resave them under the output names of SelfTimeWindows and NonSelfTimeWindows.
  2. This means that AFNI would try to take every single time window, respectively of both self vs. non-self, and copy them over and over again into the same single output files of SelfTimeWindows and NonSelfTimeWindows.

Obviously, that is nonsense. The correct code has to be the following:

directory=/users/philipp/desktop/fmri/dataset/subjects
for subject in subj1_exp1
do
cd $directory/$subject/TimeWindows
echo “Processing $subject … $SelfTimeWindows … $NonSelfTimeWindows …”

3dTcat
-prefix SelfTimeWindow
-session $directory/$subject/TimeWindows
-verb
-tr 1.0
Self21-75+tlrc Self199-253+tlrc Self427-481+tlrc Self489-543+tlrc Self549-603+tlrc Self659-713+tlrc 'Self837-891+tlrc 'Self893-947+tlrc Self955-1009+tlrc Self1013-1067+tlrc

3dTcat
-prefix NonSelfTimeWindow
-session $directory/$subject/TimeWindows
-verb
-tr 1.0
NonSelf81-135+tlrc NonSelf137-191+tlrc NonSelf257-311+tlrc NonSelf367-421+tlrc NonSelf605-659+tlrc NonSelf717-771+tlrc NonSelf779-833+tlrc NonSelf1073-1127+tlrc NonSelf1127-1179+tlrc
done

Now, this code above takes all individual (previously cut) time windows and merges them together in one run. And this is done for both self vs. nonself time windows respectively. Two runs are the output of this code: 1 that merged all self-trials together, and 1 that merged all nonself trials together.

However, the problem with the square brackets remains. I could continue from here on, but it would be nicer if the previous step of cutting the time windows would actually work with Zsh while not using square brackets. Or, if there is another way around this, how to keep square brackets in the outpit filenames but being able to open and process them later in AFNI while using Zsh.

Thanks everybody,

Philipp

3

Hi Philipp,

The basic problem here is that you are creating datasets with wildcard characters in the names (the []). That is similar to having ‘*’ or ‘?’ in the file names. Any future access would need to be quoted. Also, the square brackets come after the +tlrc or .HEAD or .BRIK.

Consider just using the numbers as you did in in the second case to go in the file names, such as in:


for SelfTimeWindows (21-75 199-253 427-481 489-543 549-603 659-713 \
                     837-891 893-947 955-1009 1013-1067 ) 
do

   cd $directory/$subject/TimeWindows
   echo "Processing $subject ... $SelfTimeWindows ... $NonSelfTimeWindows ..."

   3dTcat                                       \
      -prefix SelfTimeWindows$SelfTimeWindows   \
      -session $directory/$subject/TimeWindows  \
      -verb                                     \
      -tr 1.0                                   \
      Self+tlrc"[$SelfTimeWindows]"

done

Then you can put them together at the end.

And to throw one other example out there, consider also:

3dTcat -prefix SelfWin Self+tlrc"[21-75,199-253,427-481,489-543,549-603,659-713,837-891,893-947,955-1009,1013-1067]"

Does that seem reasonable?

  • rick
4

P.S. Aside from the loop construct (for, do and done), this syntax could probably apply to tcsh, bash or zsh.

Also, I did not actually create data to run this example with, so please take it with a grain of salt. It does run with just “echo 3dTcat …” though.

  • rick
5

Dear Rick,

thank you for taking your time again in order to help me here. I just tried your slightly changed code and it worked via Zsh! No errors or problems occured. And I didn’t know that square brackets are treated just like wildcards by AFNI. I knew about the * and ? wildcard option. But, as I said, I did not know that the same applies for the square brackets. And this is why I have to add them again for the input filename, i.e., at the end of the code for 3dTcat.

Thank you and see you around the board,

Philipp

6

It’s great to hear that it is working.
Note that wildcards are shell functionality, separate from AFNI (though AFNI does some internal processing, too).
They are handled by bash/tcsh/zsh, for example.

Thanks for the update,

  • rick