Running proc.py script over HPC server

AFNI version info (afni -ver):
23.1

code text  # or delete if not needed

Hello AFNI users!
I am trying to run the following code (proc.py script) over a HPC server:

#!/bin/tcsh


#Set up afni_proc.py

#add the path to where your individual subject folders are located below between the single quotes

setenv expdir '/work/dietschlab/rahulk/Strength_Skill_NP'
setenv subdir ${expdir}
setenv resdir ${subdir}/results

mkdir ${resdir}

#add your subject numbers separated by a space
foreach subject (TC001_Baseline)

cd ${subdir}/${subject}

afni_proc.py \
  -script ${expdir}/proc.${subject}_swallow_block \
  -scr_overwrite \
  -subj_id ${subject} \
  -out_dir ${expdir}/${subject}.results_swallow.block \
  -dsets ${subdir}/${subject}/${subject}_swallow1_epi.nii.gz \
    ${subdir}/${subject}/${subject}_swallow2_epi.nii.gz \
    ${subdir}/${subject}/${subject}_swallow3_epi.nii.gz \
  -copy_anat ${subdir}/${subject}/${subject}_t1.nii.gz \
  -anat_has_skull yes \
  -blocks despike tshift align tlrc volreg blur mask scale regress \
  -despike_new yes \
  -tlrc_base MNI152_T1_2009c+tlrc \
  -tlrc_NL_warp \
  -align_opts_aea \
  -giant_move \
  -cost lpc+ZZ \
  -volreg_align_to MIN_OUTLIER \
  -volreg_tlrc_warp \
  -volreg_align_e2a \
  -blur_in_automask \
  -regress_stim_times \
    ${subdir}/stimfiles/swallow_run.1D \
  -regress_stim_labels \
    SWALLOW \
  -regress_basis \
    'BLOCK(20,1)' \
  -mask_apply anat \
  -regress_motion_per_run \
  -test_stim_files no \
  -regress_opts_3dD \
    -GOFORIT 8 \
    -jobs 6 \
  -regress_censor_motion 0.3 \
  -regress_apply_mot_types demean deriv \
  -regress_censor_first_trs 3 \
  -regress_est_blur_errts \
  -remove_preproc_files \
  -execute

end

I am getting the following issue:

++ 3dLocalstat: AFNI version=AFNI_99.99.99 (Jun 22 2023) [64-bit]
++ Authored by: Emperor Zhark
** AFNI can't deal with 5 dimensional NIfTI(/lustre/work/dietschlab/rahulk/Strength_Skill_NP/TC001_Baseline.results_swallow.block/vlines.pb00.tcat/mask.nii.gz)
** FATAL ERROR: Can't open dataset 'mask.nii.gz'
** Program compile date = Jun 22 2023
++ 3dcalc: AFNI version=AFNI_99.99.99 (Jun 22 2023) [64-bit]
++ Authored by: A cast of thousands
** FATAL ERROR: can't open dataset tmp.mask.col.count.nii.gz
** Program compile date = Jun 22 2023

-- detrend -polort 5, new eset = ts.1.det.r01.nii.gz
++ 3dTproject: AFNI version=AFNI_99.99.99 (Jun 22 2023) [64-bit]
++ Authored by: Cox the Algebraic (Linear)
*+ WARNING:   If you are performing spatial transformations on an oblique dset,
  such as /lustre/work/dietschlab/rahulk/Strength_Skill_NP/TC001_Baseline.results_swallow.block/vlines.pb00.tcat/ts.0.orig.r01.nii.gz,
  or viewing/combining it with volumes of differing obliquity,
  you should consider running:
     3dWarp -deoblique
  on this and  other oblique datasets in the same session.
 See 3dWarp -help for details.
++ Oblique dataset:/lustre/work/dietschlab/rahulk/Strength_Skill_NP/TC001_Baseline.results_swallow.block/vlines.pb00.tcat/ts.0.orig.r01.nii.gz is 9.303011 degrees from plumb.
++ Setting up regressors
++ 1 Blocks * 6 polynomials -- 6 polort regressors
++ 646 retained time points MINUS 6 regressors ==> 640 D.O.F. left
++ no -mask option ==> processing all 359856 voxels in dataset
*+ WARNING: 3dTproject input data :: 4285 vectors are constant

++ 3dTstat: AFNI version=AFNI_99.99.99 (Jun 22 2023) [64-bit]
++ Authored by: KR Hammett & RW Cox
*+ WARNING:   If you are performing spatial transformations on an oblique dset,
  such as /lustre/work/dietschlab/rahulk/Strength_Skill_NP/TC001_Baseline.results_swallow.block/vlines.pb00.tcat/ts.1.det.r01.nii.gz,
  or viewing/combining it with volumes of differing obliquity,
  you should consider running:
     3dWarp -deoblique
  on this and  other oblique datasets in the same session.
 See 3dWarp -help for details.
++ Oblique dataset:/lustre/work/dietschlab/rahulk/Strength_Skill_NP/TC001_Baseline.results_swallow.block/vlines.pb00.tcat/ts.1.det.r01.nii.gz is 9.303011 degrees from plumb.
++ Output dataset ./tmp.stdev.nii.gz
++ 3dcalc: AFNI version=AFNI_99.99.99 (Jun 22 2023) [64-bit]
++ Authored by: A cast of thousands
++ Output dataset ./var.0.orig.r01.nii.gz
** FATAL ERROR: Can't open dataset 'mask.nii.gz'
** Program compile date = Jun 22 2023

Thanks!

  • RK

Hi-

I don't see that running on an HPC should affect anything. We use AFNI on the NIH Biowulf cluster (our local HPC). Do you both start it on that system and let it run there?

It is a bit odd to see so many AFNI version numbers in the output code having ver=99.99.99, as here:

++ 3dcalc: AFNI version=AFNI_99.99.99 (Jun 22 2023) [64-bit]

Could you please include the full afni -ver output? The one posted is not the full output format.

I am not quite sure what is happening here; I took your command and ran it similarly on one of our Bootcamp datasets (leaving out the task part) and it ran OK. It might help to see the full output that was capture in a text log---I will email you about sending that.

--pt