foreach loop errors

AFNI Message Board
1

Hi everyone,

I’m hoping you can help me with what I thought would be a simple tcsh script to loop through my subjects and run 3dmasksave and 3dROIstats on each. Here’s my script:


#!/bin/tcsh
module load pkgsrc/2018Q3
module load pkgsrc/gcc-base
module load /directoryofafniprogram/07.17.2019

set datapath = /abunchofdirectories/
set output = /abunchofdirectories/outputdirectory 
set sublist = (`cat subslist.txt`)

foreach sub ($sublist)
	echo "You are on sub $sub"
	3dmaskave -mask dACC_sigmask.nii ${datapath}/BETA_Subject${sub}_Condition001_Measure002_Component001.nii > ${output}/sub${sub}_day1_dACC_beta.txt
	3dmaskave -mask dACC_sigmask.nii ${datapath}/BETA_Subject${sub}_Condition003_Measure002_Component001.nii > ${output}/sub${sub}_6mo_dACC_beta.txt
	3dROIstats -mask dACC_sigmask.nii 'final_PCLReexp_LME.nii[8]' > ${output}/sub${sub}_ROIstat.1D
	echo "sub $sub text file saved"
end

I am getting errors trying to set and loop through the sublist text file. For reference, the text file contents looks like this:


001
002
003
...

From the above script, the (cat subslist.txt) gives me the error:

set: variable name must begin with a letter.

I have also tried loading the subslist.txt with the path name like this without changing the rest of the script and get the following error:

set sublist = abunchofdirectories/subslist.txt

foreach: Words not parenthesized.

These are the main two errors I am getting, and I have tried just about every combination of parentheses, double quotes, single quotes, back quotes, etc. that I can think of and none of them seem to work. I have looked on other forum posts and the AFNI Unix guide and can’t tell what I’m doing differently that is causing the issue. I’m hoping there’s just something super simple that I’m not noticing, but any answers would be helpful!!

Thanks,

Kyrie Amara

2

Hi, Kyrie-

I made a simplified form of that, which can run the AFNI_data6/FT_analysis/FT/ directory of our Bootcamp data; you can run it there for comparison.

First, I made a text file kind of like yours:
The file “test_file.txt” can be made with:


echo "r1" > test_file.txt
echo "r2" >> test_file.txt
echo "r3" >> test_file.txt

Then I ran:


#!/bin/tcsh

set sublist = (`cat test_file.txt`)

foreach sub ($sublist)
	echo "You are on sub $sub"
        \ls FT*${sub}*HEAD
end

… and that worked for me without error, outputting:


You are on sub r1
FT_epi_r1+orig.HEAD
You are on sub r2
FT_epi_r2+orig.HEAD
You are on sub r3
FT_epi_r3+orig.HEAD

One thing it might be worth checking is if there are any bad characters in the text file (can often happen on a Mac, in particular)—can you run:


file_tool -show_bad_all -infiles subslist.txt

… and see if that shows any bad characters, say? That could lead to a weird set error there, I guess.

Additionally, if you change


set sublist = (`cat subslist.txt`)

to


set sublist = ( 001 002 003 )

… and things work OK, then the issue might be a weird/bad character in that subslist.txt file.

Barring that, I would probably have to share a link with you to share the script and text file, and I could look at it in more detail.

–pt

3

Thanks for the quick response! Unfortunately, neither of the suggestions you sent worked. There are zero bad characters in my script, and when I put the subject numbers in the set command as a string, it gives the error: set - variable name must begin with a letter. I can send my script and sublist text file, do you need the .nii files I’m trying to call too?

4

Howdy-

OK, I just sent you a link for uploading. If possible, having dACC_sigmask.nii and a couple of the other *.nii files would be useful.

I also just noticed something else worth commenting about in this script: within your for-loop, you are using the “>” to redirect to a text file. This will not concatenate to what exists in the file, instead overwriting it each time. Since you are in a loop, I assume that you would probably want to use “>>” to redirect with concatenation in each case, instead.

If you want to clear the file first, and then start redirecting each time (e.g., in case you re-run the script, you could use something like the following:


#!/bin/tcsh

# redirect nothing into each file, with the overwriting form of redirection; 
# effectively starts an empty file, or empties a preexisting text file
# printf does not even put a newline char at the end by default (as echo would)
printf "" > FILE1.txt
printf "" > FILE2.txt

foreach var ( ${some_list} ) 
    # at each iteration, redirect into a text file, but in "concatenation" mode
    SOMETHING >> FILE1.txt
    SOMETHING_ELSE >> FILE2.txt
end

–pt

5

Kyrie, you may have eliminated these possibilities already, but various minor departures from the expected syntax of the set command may produce this error. I do not see such departures in the script you posted here, but clearly it was edited for posting (you don’t actually have a directory called ‘abunchofdirectories’, right?).

Things that would produce such error are missing one (but not both) of the spaces around the ‘=’ sign in the set datapath or set output commands. Or doing the same in the set sublist command if parentheses were NOT used on the right (I think they are not necessary there).

Or mistakenly placing a space before trailing / in the directory name. Stuff like that.

6

Hi again,

I can’t seem to access the Box folder with either of my Microsoft accounts. I get this error:

Sorry, but we’re having trouble signing you in.

AADSTS50020: User account ‘sellnowk@uwm.edu’ from identity provider ‘https://sts.windows.net/0bca7ac3-fcb6-4efd-89eb-6de97603cf21/’ does not exist in tenant ‘National Institutes of Health’ and cannot access the application ‘box.net’(NIH BOX) in that tenant. The account needs to be added as an external user in the tenant first. Sign out and sign in again with a different Azure Active Directory user account.
Request Id: c65486f3-865a-499b-9b2a-c6c95c97fb00
Correlation Id: f5c47327-3cc2-4536-829e-bb3f8d165534
Timestamp: 2021-04-14T18:35:23Z
Message: AADSTS50020: User account ‘sellnowk@uwm.edu’ from identity provider ‘live.com’ does not exist in tenant ‘National Institutes of Health’ and cannot access the application ‘box.net’(NIH BOX) in that tenant. The account needs to be added as an external user in the tenant first. Sign out and sign in again with a different Azure Active Directory user account.

7

Ha! You are correct I do not have a directory named “abunchofdirectories” I changed it for simplicity’s sake. I have checked for those possible errors in the syntax but I don’t have any, so I don’t think that’s the issue. I am including the unedited script and additional files in the submission for ptaylor (once I have access I will upload it). Thank you both for your help!

8

Ok so I have a bit of an update, although I’m not sure if it means much. I have changed my sublist text file to be a string so it looks like this:

001 002 003 …

I’ve also added a few more printf commands (thanks for that tip) to get an idea of what it’s doing. It looks like it now can read the sublist_string.txt correctly, but now I get new errors on the foreach line for the following options:

foreach sub (${sublist})
foreach sub ${sublist}
foreach sub $sublist

error - foreach: words not parenthesized

foreach (sub) ${sublist}
foreach (sub) $sublist
foreach (sub ${sublist})
foreach (sub $sublist)

error - foreach: variable name must begin with a letter.

So I apparently need to both use parentheses and not use parentheses? How does that work?

9

Hmm, that is frustrating. I have shared a similar Box link with many people (I think even fellow UW-Milwaukee-ers), and it has worked in the past… Let me try sending another one, just in case it was a blip in the system.

You should just be able to drag+drop data into it.

–pt

10

I think it would be easier for me to try the script… Something odd is happening.

–pt

11

That time worked! I uploaded it as a .tgz file since that’s how I’ve done other submissions, but let me know if you’d rather have me upload all the individual files instead