Physiological Regressor format for ricor in afniproc.py

Jshelp2 · June 5, 2025, 2:27pm

AFNI version info (afni -ver): AFNI_25.1.03

Hi everyone,

I've been on the search to find a way to get regressors for the cardiac and respiratory data from the fMRI set-up we collected data with. The PhysIO toolbox was recommended to me, which outputted data that I'm unsure would be compatible with the ricor command in afniproc.py (which is what I use for preprocessing).

The regressors are organized like this for one run (so you can visualize it):

Opening it in excel it is organized like this:

and in the txt format it looks like this:

Would this format work in afniproc.py? Also, since the preprocessing is done with all 4 runs of the experiment, I'm assuming I would have to combine the txt regressor files of each run into a single txt file of all the regressors?

Any help is appreciated! Thank you!

rickr · June 5, 2025, 3:16pm

Hello,

Since these do not appear to be slice-based regressors (we usually apply different RETROICOR regressors per slice), they need only be passed on to the regression model. This would probably be done using afni_proc.py -regress_extra_ortvec (and _labels). They would not be applied with a ricor block and -ricor_regs because those regressors are assumed to be slicewise.

Those regressors should have the same number of time points as after removal of pre-steady state time (-tcat_remove_first_trs), since they would go directly into the final regression.
Assuming you will use regressors for each run separately, they should be zero-padded for the runs they do not occupy. For this, consider "1d_tool.py -pad_into_many_runs". See examples 4a and 4b from the -help output.

-rick

Jshelp2 · August 7, 2025, 4:02pm

Hi,

Just coming back to this task now! I tried to run afni_proc.py adding the new regressors from the physiological data with the "-regress_extra_ortvec" command and the regress files zero padded for the runs that they don't contain information for like you suggested. But I got these errors during 3dDeconvolve that I was hoping someone could explain to me.

This is my afni_proc.py code:

afni_proc.py /
-subj_id 8334_phys /
-script proc.8334_phys_subj /
-scr_overwrite /
-blocks tshift align tlrc volreg mask blur scale regress /
-dsets Oddball?_8334.nii -copy_anat anatSS.sub-8334.nii /
-volreg_align_to MIN_OUTLIER /
-volreg_align_e2a /
-volreg_tlrc_warp /
-blur_size 4.0 /
-tlrc_base MNI152_2009_template.nii.gz /
-tlrc_NL_warp /
-tlrc_NL_warped_dsets anatQQ.sub-8334.nii anatQQ.sub-8334.aff12.1D anatQQ.sub-8334_WARP.nii /
-anat_has_skull no /
-volreg_compute_tsnr yes /
-radial_correlate_blocks tcat volreg regress /
-align_unifize_epi local /
-align_opts_aea /
-giant_move -cost lpc+ZZ /
-check_flip /
-mask_epi_anat yes /
-regress_extra_ortvec regress_psychophys_8334_Run1_filled.txt regress_psychophys_8334_Run2_filled.txt regress_psychophys_8334_Run3_filled.txt regress_psychophys_8334_Run4_filled.txt /
-regress_stim_times Stand_Regress_Correct_Onset_NEU_8334.txt Stand_Regress_Correct_Onset_NEG_8334.txt Odd_Regress_Correct_Onset_NEU_8334.txt Odd_Regress_Correct_Onset_NEG_8334.txt Odd_Regress_Incorrect_Onset_NEG_8334.txt Odd_Regress_Incorrect_Onset_NEU_8334.txt /
-regress_stim_labels StandardNeuC StandardNegC OddballNeuC OddballNegC OddballNegX OddballNeuX /
-regress_basis 'BLOCK(2,1)' /
-regress_opts_3dD /
-gltsym 'SYM: OddballNeuC -StandardNeuC' /
-glt_label 1 OddNeu-StandNeu /
-gltsym 'SYM: OddballNegC -StandardNegC' /
-glt_label 2 OddNeg-StandNeg /
-regress_motion_per_run /
-regress_censor_motion 0.3  /
-regress_censor_outliers 0.05 /
-regress_make_ideal_sum sum_ideal.1D /
-regress_est_blur_epits /
-regress_est_blur_errts /
-execute

These are the errors I got:

*+ WARNING: '-stim_times 1' (LOCAL) run#4 has 2 times outside range 0 .. 374 [PSFB syndrome]
*+ WARNING: '-stim_times 2' (LOCAL) run#3 has 4 times outside range 0 .. 367 [PSFB syndrome]
*+ WARNING: '-stim_times 6' (LOCAL) run#3 has 1 times outside range 0 .. 367 [PSFB syndrome]
*+ WARNING: !! '-stim_times 6' file 'stimuli/Odd_Regress_Incorrect_Onset_NEU_8334.txt' has no good stimulus time values
*+ WARNING: -stim_file 7: file length is 1612, longer than expected 1539 (from dataset)
*+ WARNING: -stim_file 8: file length is 1612, longer than expected 1539 (from dataset)
*+ WARNING: -stim_file 9: file length is 1612, longer than expected 1539 (from dataset)
*+ WARNING: -stim_file 10: file length is 1612, longer than expected 1539 (from dataset)
*+ WARNING: -stim_file 11: file length is 1612, longer than expected 1539 (from dataset)
*+ WARNING: -stim_file 12: file length is 1612, longer than expected 1539 (from dataset)
*+ WARNING: -stim_file 13: file length is 1612, longer than expected 1539 (from dataset)
*+ WARNING: -stim_file 14: file length is 1612, longer than expected 1539 (from dataset)
*+ WARNING: -stim_file 15: file length is 1612, longer than expected 1539 (from dataset)
*+ WARNING: -stim_file 16: file length is 1612, longer than expected 1539 (from dataset)
*+ WARNING: -stim_file 17: file length is 1612, longer than expected 1539 (from dataset)
*+ WARNING: -stim_file 18: file length is 1612, longer than expected 1539 (from dataset)
*+ WARNING: -stim_file 19: file length is 1612, longer than expected 1539 (from dataset)
*+ WARNING: -stim_file 20: file length is 1612, longer than expected 1539 (from dataset)
*+ WARNING: -stim_file 21: file length is 1572, longer than expected 1539 (from dataset)
*+ WARNING: -stim_file 22: file length is 1572, longer than expected 1539 (from dataset)
*+ WARNING: -stim_file 23: file length is 1572, longer than expected 1539 (from dataset)
*+ WARNING: -stim_file 24: file length is 1572, longer than expected 1539 (from dataset)
*+ WARNING: -stim_file 25: file length is 1572, longer than expected 1539 (from dataset)
*+ WARNING: -stim_file 26: file length is 1572, longer than expected 1539 (from dataset)
*+ WARNING: -stim_file 27: file length is 1572, longer than expected 1539 (from dataset)
*+ WARNING: -stim_file 28: file length is 1572, longer than expected 1539 (from dataset)
*+ WARNING: -stim_file 29: file length is 1572, longer than expected 1539 (from dataset)
*+ WARNING: -stim_file 30: file length is 1572, longer than expected 1539 (from dataset)
*+ WARNING: -stim_file 31: file length is 1572, longer than expected 1539 (from dataset)
*+ WARNING: -stim_file 32: file length is 1572, longer than expected 1539 (from dataset)
*+ WARNING: -stim_file 33: file length is 1572, longer than expected 1539 (from dataset)
*+ WARNING: -stim_file 34: file length is 1572, longer than expected 1539 (from dataset)
*+ WARNING: input stimulus time series file ortvec:2,0 is too short:
            length = 1472, but should be at least 1539.
*+ WARNING: input stimulus time series file ortvec:2,1 is too short:
            length = 1472, but should be at least 1539.
*+ WARNING: input stimulus time series file ortvec:2,2 is too short:
            length = 1472, but should be at least 1539.
*+ WARNING: input stimulus time series file ortvec:2,3 is too short:
            length = 1472, but should be at least 1539.
*+ WARNING: input stimulus time series file ortvec:2,4 is too short:
            length = 1472, but should be at least 1539.
*+ WARNING: input stimulus time series file ortvec:2,5 is too short:
            length = 1472, but should be at least 1539.
*+ WARNING: input stimulus time series file ortvec:2,6 is too short:
            length = 1472, but should be at least 1539.
*+ WARNING: input stimulus time series file ortvec:2,7 is too short:
            length = 1472, but should be at least 1539.
*+ WARNING: input stimulus time series file ortvec:2,8 is too short:
            length = 1472, but should be at least 1539.
*+ WARNING: input stimulus time series file ortvec:2,9 is too short:
            length = 1472, but should be at least 1539.
*+ WARNING: input stimulus time series file ortvec:2,10 is too short:
            length = 1472, but should be at least 1539.
*+ WARNING: input stimulus time series file ortvec:2,11 is too short:
            length = 1472, but should be at least 1539.
*+ WARNING: input stimulus time series file ortvec:2,12 is too short:
            length = 1472, but should be at least 1539.
*+ WARNING: input stimulus time series file ortvec:2,13 is too short:
            length = 1472, but should be at least 1539.
*+ WARNING: input stimulus time series file ortvec:3,0 is too short:
            length = 1472, but should be at least 1539.
*+ WARNING: input stimulus time series file ortvec:3,1 is too short:
            length = 1472, but should be at least 1539.
*+ WARNING: input stimulus time series file ortvec:3,2 is too short:
            length = 1472, but should be at least 1539.
*+ WARNING: input stimulus time series file ortvec:3,3 is too short:
            length = 1472, but should be at least 1539.
*+ WARNING: input stimulus time series file ortvec:3,4 is too short:
            length = 1472, but should be at least 1539.
*+ WARNING: input stimulus time series file ortvec:3,5 is too short:
            length = 1472, but should be at least 1539.
*+ WARNING: input stimulus time series file ortvec:3,6 is too short:
            length = 1472, but should be at least 1539.
*+ WARNING: input stimulus time series file ortvec:3,7 is too short:
            length = 1472, but should be at least 1539.
*+ WARNING: input stimulus time series file ortvec:3,8 is too short:
            length = 1472, but should be at least 1539.
*+ WARNING: input stimulus time series file ortvec:3,9 is too short:
            length = 1472, but should be at least 1539.
*+ WARNING: input stimulus time series file ortvec:3,10 is too short:
            length = 1472, but should be at least 1539.
*+ WARNING: input stimulus time series file ortvec:3,11 is too short:
            length = 1472, but should be at least 1539.
*+ WARNING: input stimulus time series file ortvec:3,12 is too short:
            length = 1472, but should be at least 1539.
*+ WARNING: input stimulus time series file ortvec:3,13 is too short:
            length = 1472, but should be at least 1539.
*+ WARNING: -------------------------------------------------
*+ WARNING: Problems with the X matrix columns, listed below:
*+ WARNING: !! * Column 21 [OddballNeuX#0] is all zeros
*+ WARNING: -------------------------------------------------
*+ WARNING: !! in Signal+Baseline matrix:
 * Largest singular value=2.21851
 * 1 singular value is less than cutoff=2.21851e-07
 * Implies strong collinearity in the matrix columns! 
*+ WARNING: !! in Signal-only matrix:
 * Largest singular value=1.22888
 * 1 singular value is less than cutoff=1.22888e-07
 * Implies strong collinearity in the matrix columns! 
** ERROR: !! 3dDeconvolve: Can't run past 4 matrix warnings without '-GOFORIT 4'
** ERROR: !!                Currently at -GOFORIT 0
** ERROR: !! See file 3dDeconvolve.err for all WARNING and ERROR messages !!
** ERROR: !! Be sure you understand what you are doing before using -GOFORIT !!
** ERROR: !! If in doubt, consult with someone or with the AFNI message board !!
** FATAL ERROR: !! 3dDeconvolve (regretfully) shuts itself down !!

Any help is appreciated. Thank you!

Jade

rickr · August 7, 2025, 7:52pm

Hi Jade,

These warnings are separate:

*+ WARNING: '-stim_times 1' (LOCAL) run#4 has 2 times outside range 0 .. 374 [PSFB syndrome]
*+ WARNING: '-stim_times 2' (LOCAL) run#3 has 4 times outside range 0 .. 367 [PSFB syndrome]
*+ WARNING: '-stim_times 6' (LOCAL) run#3 has 1 times outside range 0 .. 367 [PSFB syndrome]
*+ WARNING: !! '-stim_times 6' file 'stimuli/Odd_Regress_Incorrect_Onset_NEU_8334.txt' has no good stimulus time values

Based on the TR and number of time points, the durations of runs 3 and 4 are 367s and 374 s, respectively. But for example, stim_times 1 (StandardNeuC) has 2 stimulus onset times that are larger than t=367s. Similar issues hold with StandardNegC and OddballNeuX.

And strangely, the warning about "no good stimulus time values" for OddballNeuX suggests that the only onset time it has is after the 367s run length.

Then the warning of:

*+ WARNING: -stim_file 7: file length is 1612, longer than expected 1539 (from dataset)
...

suggests that regress_psychophys_8334_Run1_filled.txt has 1612 time points, but the EPI data has only 1539. So it is important to be sure that the physio timing is synchroninzed with the EPI timing (no pre-steady state, starts at time=0, is of the same number of time points). Look closely at the phsio data, especially any json file, to see how the timing should compare with the EPI data.

And regress_psychophys_8334_Run2_filled.txt seems to have similar problems, while the next two extra_ortvec files are too short.

It takes some care to be sure the physio output matches the EPI data, and it is possible that the original physio files do not make that clear.

-rick

Jshelp2 · August 12, 2025, 3:13pm

Hi Rick,

Thank you for your help. I was just wondering if the size of the files (either being too long or too short) would have anything to do with the zero padding I added. Or is the length only looking at the data for that particular run and ignoring the zeros in place for the other runs?

Thank you!

Best,
Jade

rickr · August 12, 2025, 8:56pm

Hi Jade,

That is only looking at the total number of time points across all runs together, where the number of time points in the individual runs is not directly considered. But if you have to check them at all, it would be good to make sure that each run has the correct number of values, since they seem to vary.

-rick

Jshelp2 · August 13, 2025, 2:19pm

Hi Rick,

I was able to run afni_proc.py without 3Ddeconvolve crashing. While I still got the same errors regarding stimulus time series being either too short or too long it did finish its execution. Im just wondering if the timing offset looks like it is giving me incorrect results. I've included the X matrix here:

Thank you so much!

Jade

Jshelp2 · August 13, 2025, 2:45pm

Hi again,

I also just realized that when I ran 1d_tool.py to zero pad the single run regressors into 4 runs that I did not adjust the run lengths (because each run has a different length). So I tried to run 1d_tool.py with this added:

 1d_tool.py /
-infile multiple_regressors_8334_Run1.txt /
-pad_into_many_runs 4 4 /
-set_run_lengths 403 393 368 375 /
-write regress_psychophys_8334_Run1_filled1.txt

and I'm getting this error:

   ** cannot pad into many runs, nt (403) != rlens[3] (375)

Those are the correct TR lengths so I'm not sure what the problem would be.

Thank you!

Jade

rickr · August 13, 2025, 2:54pm

Hi Jade,

Indeed, the X-matrix seems to show that runs 1 and 2 overlap, and maybe even 2 and 3 slightly, with a gap before 4.

The updated 1d_tool.py command might need a 4 changed to a 1. Since the file says "Run1" in it, it should probably be placed as run 1 of 4, so using the option:

-pad_into_many_runs 1 4

In this case, the input file should be of length 403, as that is what you have specified for NT in run 1.

-rick

rickr · August 13, 2025, 3:01pm

And note that for the other 3 files, that option parameter would increment. In tcsh syntax:

foreach run ( 1 2 3 4 )
    1d_tool.py                                           /
        -infile multiple_regressors_8334_Run${run}.txt   / 
        -pad_into_many_runs ${run} 4                     /
        -set_run_lengths 403 393 368 375                 /
        -write regress_psychophys_8334_Run${run}_filled.txt
end

Here I used the suffix "filled.txt" rather than "filled1.txt", since the run number is already in the name. Note that there are 3 uses of ${run}, the input filename, the first parameter to -pad_into_many_runs, and the output filename.

Does that seem reasonable?

-rick