afni_proc.py -regress_extra_ortvec after fMRIprep

Lauren_L · August 12, 2025, 7:59pm

AFNI version info (afni -ver): Precompiled binary linux_openmp_64: Jan 11 2024 (Version AFNI_24.0.01 'Caracalla')

Hi AFNI Gurus,

I'm trying to use afni_proc.py for mask, scaling and regression after fMRIprep. I would like to include a white matter and CSF nuisance regressor with the -regress_extra_ortvec option but it is currently erroring out. Here is my code:

afni_proc.py -subj_id sNTD009 -script proc.sNTD009 -scr_overwrite -out_dir                                                                                   \
      /data/ddtrp/TP2/AFNI_Subject_Data_Analyses/afni_proc_fmriprep_wm_csf_3cond                                                                               \
      -blocks mask scale regress -copy_anat                                                                                                                    \
      /data/ddtrp/TP2/NTD009/NTD009_2/00058618/derivatives/fmriprep/sub-NTD009/ses-2/anat/sub-NTD009_ses-2_space-MNI152NLin6Asym_res-2_desc-preproc_T1w.nii.gz \
      -dsets                                                                                                                                                   \
      /data/ddtrp/TP2/NTD009/NTD009_2/00058618/ica_aroma/out/notme1/denoised_func_data_nonaggr.nii.gz                                                          \
      /data/ddtrp/TP2/NTD009/NTD009_2/00058618/ica_aroma/out/notme2/denoised_func_data_nonaggr.nii.gz                                                          \
      /data/ddtrp/TP2/NTD009/NTD009_2/00058618/ica_aroma/out/notme3/denoised_func_data_nonaggr.nii.gz                                                          \
      -tcat_remove_first_trs 0 -mask_apply group -mask_dilate 6                                                                                                \
      -regress_stim_times_offset 1.153182 -regress_stim_times                                                                                                  \
      /data/ddtrp/TP2/NotMeStimTimes/StimTime1Ds/island_2025-08-04_16.01.25/st/sNTD009_Famous_Female_times.1D                                                  \
      /data/ddtrp/TP2/NotMeStimTimes/StimTime1Ds/island_2025-08-04_16.01.25/st/sNTD009_Other_Female_times.1D                                                   \
      /data/ddtrp/TP2/NotMeStimTimes/StimTime1Ds/island_2025-08-04_16.01.25/st/sNTD009_Self_Female_times.1D                                                    \
      -regress_stim_labels Famous_Female_times Other_Female_times                                                                                              \
      Self_Female_times -regress_basis 'BLOCK(23.6,1)' -regress_make_cbucket                                                                                   \
      yes -regress_extra_ortvec                                                                                                                                \
      /data/ddtrp/TP2/Nuisance_Reg_WM_CSF/NTD009_csf_notme1.1D                                                                                                 \
      -regress_extra_ortvec                                                                                                                                    \
      /data/ddtrp/TP2/Nuisance_Reg_WM_CSF/NTD009_csf_notme2.1D                                                                                                 \
      -regress_extra_ortvec                                                                                                                                    \
      /data/ddtrp/TP2/Nuisance_Reg_WM_CSF/NTD009_csf_notme3.1D                                                                                                 \
      -regress_extra_ortvec                                                                                                                                    \
      /data/ddtrp/TP2/Nuisance_Reg_WM_CSF/NTD009_wm_notme1.1D                                                                                                  \
      -regress_extra_ortvec                                                                                                                                    \
      /data/ddtrp/TP2/Nuisance_Reg_WM_CSF/NTD009_wm_notme2.1D                                                                                                  \
      -regress_extra_ortvec                                                                                                                                    \
      /data/ddtrp/TP2/Nuisance_Reg_WM_CSF/NTD009_wm_notme3.1D                                                                                                  \
      -regress_extra_ortvec_labels CSF1 CSF2 CSF3 WM1 WM2 WM3                                                                                                  \
      -regress_3dD_stop -regress_reml_exec -regress_opts_3dD -jobs 2 -bout                                                                                     \
      -num_glt 3 -gltsym 'SYM: Self_Female_times -Other_Female_times'                                                                                          \
      -glt_label 1 Self_F-Other_F -gltsym 'SYM: Self_Female_times                                                                                              \
      -Famous_Female_times' -glt_label 2 Self_F-Famous_F -gltsym 'SYM:                                                                                         \
      Famous_Female_times -Other_Female_times' -glt_label 3 Famous_F-Other_F                                                                                   \
      -regress_compute_fitts -regress_est_blur_errts -regress_local_times

This code throws the following error:

-- applying input view as +tlrc
** have 1 extra ortvec but 6 extra ort labels
** invalid block : regress
----------------------------------------------------------------------
** failed command (create_blocks):

etc.

My task ("NotMe") has three runs:

-dsets
/data/ddtrp/TP2/NTD010/NTD010_2/00068791/ica_aroma/out/notme1/denoised_func_data_nonaggr.nii.gz
/data/ddtrp/TP2/NTD010/NTD010_2/00068791/ica_aroma/out/notme2/denoised_func_data_nonaggr.nii.gz
/data/ddtrp/TP2/NTD010/NTD010_2/00068791/ica_aroma/out/notme3/denoised_func_data_nonaggr.nii.gz

I currently have my white matter and csf 1d files separated by run:
-regress_extra_ortvec
/data/ddtrp/TP2/Nuisance_Reg_WM_CSF/NTD010_csf_notme1.1D
-regress_extra_ortvec
/data/ddtrp/TP2/Nuisance_Reg_WM_CSF/NTD010_csf_notme2.1D
-regress_extra_ortvec
/data/ddtrp/TP2/Nuisance_Reg_WM_CSF/NTD010_csf_notme3.1D
-regress_extra_ortvec
/data/ddtrp/TP2/Nuisance_Reg_WM_CSF/NTD010_wm_notme1.1D
-regress_extra_ortvec
/data/ddtrp/TP2/Nuisance_Reg_WM_CSF/NTD010_wm_notme2.1D
-regress_extra_ortvec
/data/ddtrp/TP2/Nuisance_Reg_WM_CSF/NTD010_wm_notme3.1D
-regress_extra_ortvec_labels CSF1 CSF2 CSF3 WM1 WM2 WM3

Do I need to set up the white matter/CSF 1D files differently (e.g., combined across runs and one file per type e.g., CSF, WM?) and then have the labels just be CSF WM ? Does the fact that I've designated the -dsets as 3 separate files (1 per run) impact how the -regress_extra_ortvec files should be setup so they match?

Thanks for your time and help!
Best,
Lauren

ptaylor · August 12, 2025, 8:55pm

Hi, Lauren-

I just spaced the original command out a bit more, with vertical alignment (introducing variables to shrink the paths, NB: this is tcsh syntax but you could switch to any other shell syntax):

set p1 = /data/ddtrp/TP2
set p2 = ${p1}/NTD009/NTD009_2/00058618/derivatives/fmriprep/sub-NTD009/ses-2
set p3 = ${p1}/Nuisance_Reg_WM_CSF
set p4 = ${p1}/NotMeStimTimes/StimTime1Ds/island_2025-08-04_16.01.25/st
set p5 = ${p1}/NTD009/NTD009_2/00058618/ica_aroma/out
set p6 = ${p1}/AFNI_Subject_Data_Analyses

afni_proc.py                                                                 \
    -subj_id                      sNTD009                                    \
    -script                       proc.sNTD009                               \
    -scr_overwrite                                                           \
    -out_dir                      ${p6}/afni_proc_fmriprep_wm_csf_3cond      \
    -blocks                       mask scale regress                         \
    -copy_anat                    ${p2}/anat/sub-NTD009_ses-2_space-MNI152NLin6Asym_res-2_desc-preproc_T1w.nii.gz \
    -dsets                        ${p5}/notme1/denoised_func_data_nonaggr.nii.gz \
                                  ${p5}/notme2/denoised_func_data_nonaggr.nii.gz \
                                  ${p5}/notme3/denoised_func_data_nonaggr.nii.gz \
    -tcat_remove_first_trs        0                                          \
    -mask_apply                   group                                      \
    -mask_dilate                  6                                          \
    -regress_stim_times_offset    1.153182                                   \
    -regress_stim_times           ${p4}/sNTD009_Famous_Female_times.1D       \
                                  ${p4}/sNTD009_Other_Female_times.1D        \
                                  ${p4}/sNTD009_Self_Female_times.1D         \
    -regress_stim_labels          Famous_Female_times Other_Female_times     \
                                  Self_Female_times                          \
    -regress_basis                'BLOCK(23.6,1)'                            \
    -regress_make_cbucket         yes                                        \
    -regress_extra_ortvec         ${p3}/NTD009_csf_notme1.1D                 \
    -regress_extra_ortvec         ${p3}/NTD009_csf_notme2.1D                 \
    -regress_extra_ortvec         ${p3}/NTD009_csf_notme3.1D                 \
    -regress_extra_ortvec         ${p3}/NTD009_wm_notme1.1D                  \
    -regress_extra_ortvec         ${p3}/NTD009_wm_notme2.1D                  \
    -regress_extra_ortvec         ${p3}/NTD009_wm_notme3.1D                  \
    -regress_extra_ortvec_labels  CSF1 CSF2 CSF3 WM1 WM2 WM3                 \
    -regress_3dD_stop                                                        \
    -regress_reml_exec                                                       \
    -regress_opts_3dD             -jobs 2                                    \
                                  -bout                                      \
                                  -num_glt 3                                 \
                                  -gltsym                                    \
                                  'SYM: Self_Female_times -Other_Female_times' \
                                  -glt_label 1 Self_F-Other_F                \
                                  -gltsym                                    \
                                  'SYM: Self_Female_times -Famous_Female_times' \
                                  -glt_label 2 Self_F-Famous_F               \
                                  -gltsym                                    \
                                  'SYM: Famous_Female_times -Other_Female_times' \
                                  -glt_label 3 Famous_F-Other_F              \
    -regress_compute_fitts                                                   \
    -regress_est_blur_errts                                                  \
    -regress_local_times

To your initial question, you should just use one -regress_extra_ortvec .. option and load in the number of regressor files you want after it, and use one -regress_extra_ortvec_labels .. call with the same number of labels, in 1:1 correspondence. Right now, you have 6 calls to the first option, the error message.

To the additional part of this, the regressors loaded in this way are supposed to be of full duration of the final regression model, so of the concatenated lengths of the 3 EPI files here. I guess that would mean zeropadding each single-run regressor here appropriately. I think 1d_tool.py can help with that. If your EPI runs are the same length, then Ex. 4a in the program help could be the way to go:

Given a file of regressors (for example) across a single run (run 2),
created a new file that is padded with zeros, so that it now spans
many (7) runs.  Runs are 1-based here.

  1d_tool.py -infile ricor_r02.1D -pad_into_many_runs 2 7 \
             -write ricor_r02_all.1D

... and Ex. 4b shows a case of runs being different lengths.

(And you aren't going to censor any motion volumes, it seems?)

--pt

ptaylor · August 12, 2025, 9:08pm

Also, I will note that if you update your AFNI version, you will be able to get more APQC HTML output for reviewing/quality control checking your data and processing (as described here), even though part of your processing was done outside afni_proc.py. That update should be available from AFNI ver 25.0.04 (but updating to the latest version would make most sense, probably).

--pt

Lauren_L · August 13, 2025, 1:44am

Thanks, Paul!
Your suggestion fixed the -regress_extra_ortvec issue.

I do have a few follow-up questions:

Mask block – I wasn’t able to get the script to run when including the mask block:

-mask_apply group -mask_dilate 6

Without the mask block, I also couldn’t get -regress_est_blur_errts to run.

Pipeline context – My initial preprocessing was done in fMRIPrep. After that, I smoothed the data and ran it through ICA-AROMA (so I’m not including motion regressors in this afni_proc.py script).
Main questions:

Is there a way to use the mask block when starting from fMRIPrep outputs?
Are there group-level analyses I can’t do without having the subject-level mask and blur estimates from the first-level analysis?

For reference, here’s the current version of my script that runs successfully (without mask and blur estimation):

# -----------------------------------------------------------------------
# Directories
# -----------------------------------------------------------------------
set top_dir    = /data/ddtrp/TP2/AFNI_Subject_Data_Analyses
set stim_dir   = /data/ddtrp/TP2/NotMeStimTimes/StimTime1Ds/island_2025-08-04_16.01.25/st
set nuisance_dir = /data/ddtrp/TP2/Nuisance_Reg_WM_CSF

# -----------------------------------------------------------------------
# Slice timing offsets for TP2
# nslices = 44 for TP2
# Trio:   TR=2.360s → 1.153182 sec
# Prisma: TR=2.390s → 1.167841 sec

#fMRIprep slice time correction (3dTshift) defaults to the middle of the TR instead of the beginning. Therefore, stim times need to be adjusted using -regress_stim_times_offset. 
# This way BOLD and stim times will match. This value is roughly your TR/2 or more precisely according to Rick Reynolds (TR/2 * (nslices-1)/nslices).
#For TP2, Subjects collected on Trio: 2.360sec/2= 1.18 ; 1.18 * (44-1/44) = 1.153182
#For TP2, Subjects collected on Prism (Note PRISMA started at NTD167+): 2.390/2=1.195 ; 1.195 * (44-1/44) =  1.167841
# -----------------------------------------------------------------------

foreach subj ($subjects_included)

    # Determine scanner type from subject ID
    set numpart = `echo $subj | sed 's/[^0-9]//g'`
    if ( $numpart >= 167 ) then
        set stim_offset = 1.167841   # Prisma
    else
        set stim_offset = 1.153182   # Trio
    endif

    # -------------------------------------------------------------------
    # Auto-locate anat and epi directories
    # -------------------------------------------------------------------
    set anat_dir = `ls -d /data/ddtrp/TP2/${subj}/${subj}_2/*/derivatives/fmriprep/sub-${subj}/ses-2/anat | head -n 1`
    set epi_dir  = `ls -d /data/ddtrp/TP2/${subj}/${subj}_2/*/ica_aroma/out | head -n 1`

    if ( "$anat_dir" == "" || "$epi_dir" == "" ) then
        echo "ERROR: Missing anat or epi dir for subject $subj"
        continue
    endif    
    
## -------------------------------------------------------------------
    # Run afni_proc.py
    # -------------------------------------------------------------------

    afni_proc.py -subj_id s{$subj}                                    					    \
         -script proc.s{$subj}						  										\
	  	 -scr_overwrite                                     								\
#	 -execute							     												\
	 -out_dir $top_dir/{$subj}_afni_proc_fmriprep_wm_csf_3cond								\
     -blocks scale regress           														\
     -copy_anat "$anat_dir/sub-${subj}_ses-2_space-MNI152NLin6Asym_res-2_desc-preproc_T1w.nii.gz" \
     -dsets $epi_dir/notme1/denoised_func_data_nonaggr.nii.gz            	                \
    	   $epi_dir/notme2/denoised_func_data_nonaggr.nii.gz         	                    \
    	   $epi_dir/notme3/denoised_func_data_nonaggr.nii.gz         	                    \
     -tcat_remove_first_trs 0                                         						\
#	 -mask_apply group						    											\
#	 -mask_dilate 6																		    \
	 -regress_stim_times_offset $stim_offset												\
     -regress_stim_times $stim_dir/s{$subj}_*Female_times.1D           					    \
     -regress_stim_labels Famous_Female_times Other_Female_times Self_Female_times   		\
     -regress_basis 'BLOCK(23.6,1)'                   					                    \
	 -regress_make_cbucket yes					    										\
	 -regress_extra_ortvec /data/ddtrp/TP2/Nuisance_Reg_WM_CSF/{$subj}_csf_notme_all.1D		\
	 					   /data/ddtrp/TP2/Nuisance_Reg_WM_CSF/{$subj}_wm_notme_all.1D		\
	 -regress_extra_ortvec_labels CSF WM													\
	 -regress_3dD_stop						      											\
     -regress_reml_exec                                                    					\
     -regress_opts_3dD						      											\
	    -jobs 2							      												\
	    -bout                                                	      						\
	    -num_glt 3							      											\
            -gltsym 'SYM: Self_Female_times -Other_Female_times'	      					\
		-glt_label 1 Self_F-Other_F            			      								\
            -gltsym 'SYM: Self_Female_times -Famous_Female_times'             				\
		-glt_label 2 Self_F-Famous_F           			      								\
            -gltsym 'SYM: Famous_Female_times -Other_Female_times'	      					\
		-glt_label 3 Famous_F-Other_F			              								\
    -regress_compute_fitts                                                					\
#    -regress_est_blur_errts						      										\
	-regress_local_times
end

Thank you so much for your time and help!

ptaylor · August 13, 2025, 2:52am

Hi, Lauren-

Glad the -regress_extra_ortvec is resolved.

For the mask block:

Yes, you can add it.

I think, however, that the "group" keyword argument there can only be used if the tlrc block is used:

A 'group' anat mask will be created if the 'tlrc' block is used
(via the -blocks or -tlrc_anat options).  In such a case, the anat
template will be made into a binary mask.

How would it be to use, say, -mask_epi_anat yes? That should be fine for estimating blur. (I think dilating a mask by 6 layers and then estimating blur within that enlarged space, which would include a lot of outside-the-brain, might give a diluted value of blur estimation, too.)
Note that the mask is estimated, but not applied to mask the data itself, because we like to see data throughout the FOV, to help understand everything better and to QC/troubleshoot. The mask will be used as necessary for individual calculations, like for blur estimation/etc. So, it's useful to have, and the above is what we would normally go for. We describe a bit more here, in the paragraph starting: <<The “mask” block leads to the estimation of...>>

Re. pipeline context and ICA-AROMA: good to know about the motion regressors. Censoring (AKA scrubbing) is a bit different, though in task analysis it might matter less, depending on the motion profiles and event duration. In your case here, it looks like a block design with long blocks, so that will be less sensitive to some incidental motion.

For group-level analyses: clustering probably needs to have a mask at hand, to define a region for blur estimation. Otherwise, most don't at the voxelwise level; some newer Bayesian programs, which use pooling, will probably require it, to pool within a reasonable volume to share information over (i.e., the brain, excluding background).

--pt

Lauren_L · August 13, 2025, 5:39pm

Thanks, Paul! I tried switching to -mask_epi_anat yes, but the spatially normalized anatomical dataset outputted by fMRIprep still has a skull so afni_proc.py exited with an error.

fMRIprep does not appear to save a skullstripped version of the normalized T1 - but it does save a binarized brain mask of the normalized anatomical. Could I somehow give afniproc that instead?

Alternatively, should I take the fMRIprep spatially normalized anatomical (e.g., sub-${subj}_ses-2_space-MNI152NLin6Asym_res-2_desc-preproc_T1w.nii.gz) and skull-strip with e.g., 3dSkullStrip and designate this dataset as the anat file in afni_proc instead?

Thanks for your time!

Lauren_L · August 13, 2025, 6:49pm

One quick follow-up. I went ahead and used 3dSkullStrip on the fMRIprep spatially normalized anatomical that still had a skull and tried inputting that as the anat dataset. But it still throws this error (even though the file does not have skull anymore):

-- applying input view as +tlrc
-- including default: -radial_correlate_blocks regress
-- including default: -find_var_line_blocks tcat
-- tcat: reps is now 99
++ updating polort to 2, from run len 233.6 s
** cannot apply -mask_epi_anat
   (anat still has skull)
** script creation failure for block 'mask'

Thanks for your help!

rickr · August 13, 2025, 9:16pm

Hi Lauren,

You should be able to tell afni_proc.py that there is no skull (it won't know on its own).

-anat_has_skull no

See if that helps.

-rick