** failed to process options... for input files with gen_ss_review_scripts.py

AFNI version info (afni -ver): (Version AFNI_23.3.07 'Septimius Severus')

Hello,

I have been working on modifying our output script from AFNI_proc.py to include all three of our tasks that we are using, this way we only use one script to do the preprocessing for all three of our tasks. I ran into an error with the gen_ss_review_scripts.py. I have specific named files for gen_ss_review_scripts.py to input for the specific tasks and when I try to have it get the tcat_dset and volreg_dset I run into a failure to process options. Below is my modified script for the gen_ss_review_scripts.py with only the Alcohol task.

gen_ss_review_scripts.py    \
	-exit0              \
	-cvar scr_basic ~$dir/{$subj}/{$subj}.all_task_output/s.basic_Alcohol \
	-cvar scr_drive ~$dir/{$subj}/{$subj}.all_task_output/s.drive_Alcohol \
	-cvar cmds_drive ~$dir/{$subj}/{$subj}.all_task_output/s.cmds_Alcohol \
	-cvar xstim ~$dir/{$subj}/{$subj}.all_task_output/x.stim.Alcohol.1D    \
	-mot_limit 0.3 -out_limit 0.05           \
	-uvar tcat_dset pb00.$subj.Alcohol.r01.tcat+orig.HEAD pb00.$subj.Alcohol.r02.tcat+orig.HEAD  \
	-uvar outlier_dset outcount_Alcohol_rall.1D   \
	-uvar enorm_dset motion_{$subj}_Alcohol_enorm.1D   \
	-uvar censor_dset motion_{$subj}_Alcohol_censor.1D \
	-uvar stats_dset stats_Alcohol.$subj+tlrc.HEAD \
	-uvar errts_dset errts_Alcohol.$subj+tlrc.HEAD  \
	-uvar motion_dset dfile_rall.Alcohol.1D \
	-uvar volreg_dset pb03.$subj.Alcohol.r01.volreg+tlrc.HEAD pb03.$subj.Alcohol.r02.volreg+tlrc.HEAD    \
	-uvar xmat_regress X.xmat_Alcohol.1D    \
	-uvar final_anat anat_final.$subj+tlrc.HEAD \
	-uvar mask_dset mask_epi_anat.Alcohol.$subj+tlrc.HEAD \
	-uvar xmat_uncensored X.nocensor_Alcohol.xmat.1D  \
	-uvar tsnr_dset TSNR.$subj.Alcohol+tlrc.HEAD  \
	-uvar gcor_dset out.gcor.Alcohol.1D   \
	-ss_review_dset out.ss_review.$subj.Alcohol.txt  \
	-write_uvars_json out.ss_review_uvars.Alcohol.json

The output failure when running this script is the following:

** SVWD: simple variable 'tcat_dset' <class 'str'>
but have list value: ['pb00.9835MMH.VIA.r01.tcat+orig.HEAD', 'pb00.9835MMH.VIA.r02.tcat+orig.HEAD']
** SVWD: simple variable 'volreg_dset' <class 'str'>
but have list value: ['pb03.9835MMH.VIA.r01.volreg+tlrc.HEAD', 'pb03.9835MMH.VIA.r02.volreg+tlrc.HEAD']
** failed to process options...

I also tried -sv instead of -uvar and got the following output failure:
** SVWD: simple variable 'out_limit' <class 'float'>

Any thoughts on this?

Thanks!

For both of those variables (tcat_dset and volreg_dset), try passing just a single dataset instead of all of the names. It should be able to get the number of runs and such from elsewhere.

  • rick

Hi Rick,

Thanks for the reply. I changed the tcat_dset and volreg_dset to only have one of the runs selected. The following script is with the change to the tcat_dset and volreg_dset.

gen_ss_review_scripts.py    \
	-exit0              \
	-cvar scr_basic ~/$dir/{$subj}/{$subj}.all_task_output/s.basic_Alcohol \
	-cvar scr_drive ~/$dir/{$subj}/{$subj}.all_task_output/s.drive_Alcohol \
	-cvar cmds_drive ~/$dir/{$subj}/{$subj}.all_task_output/s.cmds_Alcohol \
	-cvar xstim ~/$dir/{$subj}/{$subj}.all_task_output/x.stim.Alcohol.1D    \
	-mot_limit 0.3 -out_limit 0.05           \
	-uvar tcat_dset pb00.$subj.Alcohol.r01.tcat+orig.HEAD   \
	-uvar outlier_dset outcount_Alcohol_rall.1D   \
	-uvar enorm_dset motion_{$subj}_Alcohol_enorm.1D   \
	-uvar censor_dset motion_{$subj}_Alcohol_censor.1D \
	-uvar stats_dset stats_Alcohol.$subj+tlrc.HEAD \
	-uvar errts_dset errts_Alcohol.$subj+tlrc.HEAD  \
	-uvar motion_dset dfile_rall.Alcohol.1D \
	-uvar volreg_dset pb03.$subj.Alcohol.r01.volreg+tlrc.HEAD     \
	-uvar xmat_regress X.xmat_Alcohol.1D    \
	-uvar final_anat anat_final.$subj+tlrc.HEAD \
	-uvar mask_dset mask_epi_anat.Alcohol.$subj+tlrc.HEAD \
	-uvar xmat_uncensored X.nocensor_Alcohol.xmat.1D  \
	-uvar tsnr_dset TSNR.$subj.Alcohol+tlrc.HEAD  \
	-uvar gcor_dset out.gcor.Alcohol.1D   \
	-ss_review_dset out.ss_review.$subj.Alcohol.txt  \
	-write_uvars_json out.ss_review_uvars.Alcohol.json

The output failure when running this script is the following:

** warning: guess tcat, non-unique tcat list: ['9835MMH.Alcohol', '9835MMH.Food', '9835MMH.VIA']
** cannot decode_1D '' in ''
** guess rm_trs: failed to find sub-brick selector in 3dTcat
** failed to open text file '/$dir/9835MMH/9835MMH.all_task_output/s.basic_Alcohol' for writing
++ writing ss review basic: /$dir/9835MMH/9835MMH.all_task_output/s.basic_Alcohol
Traceback (most recent call last):
File "/Users/garrettl/abin/gen_ss_review_scripts.py", line 3461, in
sys.exit(main())
^^^^^^
File "/Users/garrettl/abin/gen_ss_review_scripts.py", line 3454, in main
if me.write_scripts():
^^^^^^^^^^^^^^^^^^
File "/Users/garrettl/abin/gen_ss_review_scripts.py", line 2700, in write_scripts
if self.make_drive_script(): return 1
^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/garrettl/abin/gen_ss_review_scripts.py", line 2972, in make_drive_script
if self.drive_align_check(): return 1
^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/garrettl/abin/gen_ss_review_scripts.py", line 3253, in drive_align_check
vpre = self.dsets.volreg_dset.prefix
^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'VarsObject' object has no attribute 'volreg_dset'

I also tried with having the following for tcat_dset and volreg_dset variable settings:
-uvar tcat_dset pb00.$subj.Alcohol.r*.tcat+orig.HEAD
-uvar volreg_dset pb03.$subj.Alcohol.r*.volreg+tlrc.HEAD \

Which got the following output:
** SVWD: simple variable 'tcat_dset' <class 'str'>
but have list value: ['pb00.9835MMH.Alcohol.r01.tcat+orig.HEAD', 'pb00.9835MMH.Alcohol.r02.tcat+orig.HEAD']
** SVWD: simple variable 'volreg_dset' <class 'str'>
but have list value: ['pb03.9835MMH.Alcohol.r01.volreg+tlrc.HEAD', 'pb03.9835MMH.Alcohol.r02.volreg+tlrc.HEAD']
** failed to process options...

I also ran it with the following tcat_dset and volreg_dset variable settings:
-uvar tcat_dset pb00.$subj.Alcohol.*
-uvar volreg_dset pb03.$subj.Alcohol.* \

and this got the following output:
** SVWD: simple variable 'tcat_dset' <class 'str'>
but have list value: ['pb00.9835MMH.Alcohol.r01.tcat+orig.BRIK', 'pb00.9835MMH.Alcohol.r01.tcat+orig.HEAD', 'pb00.9835MMH.Alcohol.r02.tcat+orig.BRIK', 'pb00.9835MMH.Alcohol.r02.tcat+orig.HEAD']
** SVWD: simple variable 'volreg_dset' <class 'str'>
but have list value: ['pb03.9835MMH.Alcohol.r01.volreg+tlrc.BRIK', 'pb03.9835MMH.Alcohol.r01.volreg+tlrc.HEAD', 'pb03.9835MMH.Alcohol.r02.volreg+tlrc.BRIK', 'pb03.9835MMH.Alcohol.r02.volreg+tlrc.HEAD']
** failed to process options...

Before digging too deeply here, what is the advantage of modifying the proc script and not the actual afni_proc.py command? It is far safer to have different afni_proc.py commands than to mess with the proc script and hope nothing gets broken. It is actually good that some command might fail - far worse is having the script run, but with a mistake that you don't know about.

If the tasks are very similar, then the AP command scripts should be very similar, which is great. I highly suggest leaving the proc scripts as they are.

  • rick

So for our study each of the participants have resting state and then we have them do three different tasks (Alcohol, Food and VIA). For our script we were wanting to take the proc script and put a variable in for the tasks then loop the script to run for each of the tasks, having the only difference between tasks being the nii.gz files and regressors that it uses. The thought behind modifying the script was that we want all three of our tasks to have the exact same parameters or settings for every participant. By having it in one it would allow us to more easily change the parameters or settings for all three tasks and know that all the preprocessing for each task was done the exact same way. This would prevent any errors from how the preprocessing was ran ; for example, if someone was looking at the Food task data and decided they want to try running the preprocessing a different way, then they would change the one script and this change would happen for the Alcohol and VIA task as well. This way we would be confident that everything is done the same way and someone didn't forget to change one of the scripts or the scripts didn't exactly match each other.

The main changes to the script is having it name the files that the script makes to have the task in them. So when it's running for Alcohol all the files that the script makes is being named with Alcohol in them. Same with Food and VIA. Then just have been having trouble making sure that the gen_ss_review_script is getting the correctly named files made from the script.

We do the exact same thing, except it is the afni_proc.py commands that we loop over, not the proc scripts. The afni_proc.py command script can have the same flexibility and variable name use that you describe, and then you are only messing with a simple 20-30 line file, rather than a 600+ line one.

Just put that logic into the AP command calling. Note the run/do script pairs in our (slightly outdated now) APMULTI_Demo1_rest tree. That can be installed locally by running @Install_APMULTI_Demo1_rest.

  • rick

Thank you, I will work on modifying the AFNI_proc.py command script to have it loop for each of our tasks.

-Garrett