Error in Surface-based afni_proc.py?

AFNI Message Board
1

AFNI version info (afni -ver):
Personal Laptop)
Precompiled binary macos_13_ARM_clang: May 22 2024 (Version AFNI_24.1.10 'Publius Septimius Geta')

Server)
Precompiled binary linux_ubuntu_16_64: Oct 31 2024 (Version AFNI_24.3.06 'Elagabalus')

afni_proc.py -subj_id sub-100307 -script proc.sub-100307 -scr_overwrite                                             \
      -out_dir                                                                                                        \
      /Volumes/scratch/usr_name/HCP/HCP_bids/sub-100307/../derivatives/FS/sub-100307/afniproc_output                  \
      -dsets                                                                                                          \
      /Volumes/scratch/usr_name/HCP/HCP_bids/sub-100307/func/sub-100307_task-REST_acq-LR_run-01_bold.nii.gz           \
      /Volumes/scratch/usr_name/HCP/HCP_bids/sub-100307/func/sub-100307_task-REST_acq-LR_run-02_bold.nii.gz           \
      -copy_anat                                                                                                      \
      /Volumes/scratch/usr_name/HCP/HCP_bids/sub-100307/anat/sub-100307_run-01_T1w.nii.gz                             \
      -anat_has_skull yes -anat_follower anat_w_skull anat                                                            \
      /Volumes/scratch/usr_name/HCP/HCP_bids/sub-100307/anat/sub-100307_run-01_T1w.nii.gz                             \
      -tcat_remove_first_trs 4 -blocks tshift align volreg mask surf scale                                            \
      regress -align_opts_aea -cost lpc+ZZ -giant_move -cmass cmass                                                   \
      -check_flip -volreg_align_to first -volreg_align_e2a -mask_epi_anat yes                                         \
      -mask_segment_anat yes -mask_segment_erode yes -surf_anat                                                       \
      /Volumes/scratch/usr_name/HCP/HCP_bids/sub-100307/../derivatives/FS/sub-100307/surf/SUMA/sub-100307_SurfVol.nii \
      -surf_spec                                                                                                      \
      /Volumes/scratch/usr_name/HCP/HCP_bids/sub-100307/../derivatives/FS/sub-100307/surf/SUMA/sub-100307_lh.spec     \
      /Volumes/scratch/usr_name/HCP/HCP_bids/sub-100307/../derivatives/FS/sub-100307/surf/SUMA/sub-100307_rh.spec     \
      -regress_censor_motion 0.4 -regress_censor_outliers 0.05                                                        \
      -regress_bandpass 0.01 0.1 -regress_ROI WMe CSFe                                                                \
      -regress_apply_mot_types demean deriv -regress_motion_per_run                                                   \
      -regress_opts_3dD -jobs 8 -execute

error massage
-- applying input view as +orig
** warning: removing first 4 TRs from beginning of each run
--> the stimulus timing files must reflect the removal of these TRs

** error: spec files MUST contain lh or rh, and otherwise
match (consider making copies, like SUBJ.stdmesh.lh.spec)
** invalid block : surf

The script works well in my personal laptop (Mac), but does not work at my server computer (Linux).
Since my input was sub-100307_?h.spec, I think script might misunderstand as my command did not have rh or lh. I tested it after changing my spec file name like rh_sub-100307_rh.spec and rh_sub-100307_lh.spec and changing command like rh_sub-100307_?h.spec. Then, it did work!!! I can easily recall some indirect way to solve this error, but I just want to report it :smiley:

Other questions on surface-based afni_proc.py:

  1. Is it okay to include mask between volreg and surface? Many example on the web don't have mask block, but I would like to regress out WM and CSF. So, I did like that, but I am not sure it's okay.
  2. Does afni_proc.py save the regressor vector(?) in the output folder or somewhere?
2

Hi-

For the first part about the error, I ran the following without error on my Ubuntu linux setup, with AFNI_25.0.04, from the AFNI Bootcamp directory ~/AFNI_data6/FT_analysis. It contains all your steps and syntax, just with different file names:

afni_proc.py                                                                 \
    -subj_id                  sub-TEST_AP                                     \
    -script                   proc.TEST_AP                                \
    -scr_overwrite                                                           \
    -out_dir                  /home/ptaylor/AFNI_data6/FT_analysis/TEST_AP \
    -dsets                    /home/ptaylor/AFNI_data6/FT_analysis/FT/FT_epi_r1+orig.HEAD \
                              /home/ptaylor/AFNI_data6/FT_analysis/FT/FT_epi_r2+orig.HEAD \
    -copy_anat                /home/ptaylor/AFNI_data6/FT_analysis/FT/FT_anat+orig.HEAD \
    -anat_has_skull           yes                                            \
    -anat_follower            anat_w_skull anat                              \
                              /home/ptaylor/AFNI_data6/FT_analysis/FT/FT_anat+orig.HEAD \
    -tcat_remove_first_trs    4                                              \
    -blocks                   tshift align volreg mask surf scale regress    \
    -align_opts_aea           -cost lpc+ZZ                                   \
                              -giant_move                                    \
                              -cmass cmass                                   \
                              -check_flip                                    \
    -volreg_align_to          first                                          \
    -volreg_align_e2a                                                        \
    -mask_epi_anat            yes                                            \
    -mask_segment_anat        yes                                            \
    -mask_segment_erode       yes                                            \
    -surf_anat                /home/ptaylor/AFNI_data6/FT_analysis/FT/SUMA/FT_SurfVol.nii \
    -surf_spec                /home/ptaylor/AFNI_data6/FT_analysis/FT/SUMA/std.60.FT_lh.spec \
                              /home/ptaylor/AFNI_data6/FT_analysis/FT/SUMA/std.60.FT_rh.spec \
    -regress_censor_motion    0.4                                            \
    -regress_censor_outliers  0.05                                           \
    -regress_bandpass         0.01 0.1                                       \
    -regress_ROI              WMe CSFe                                       \
    -regress_apply_mot_types  demean deriv                                   \
    -regress_motion_per_run                                                  \
    -regress_opts_3dD         -jobs 8                                        \
    -execute

Can you try that on your server and see if that gives an error?

--pt

1 Like
3

I guess you might also want to run the system check on the server, too, to make sure nothing is missing:

afni_system_check.py -check_all

--pt

1 Like
4

And also to your question about where the regressors might be stored, looking at the 3dDeconvolve command in the proc.* script can tell you the names of the regressors. When I ran the command above, it was:

    3dDeconvolve -input pb04.$subj.$hemi.r*.scale.niml.dset         \
        -censor censor_${subj}_combined_2.1D                        \
        -ortvec bandpass_rall.1D bandpass                           \
        -ortvec ROI.WMe_rall.1D ROI.WMe                             \
        -ortvec ROI.CSFe_rall.1D ROI.CSFe                           \
        -ortvec mot_demean.r01.1D mot_demean_r01                    \
        -ortvec mot_demean.r02.1D mot_demean_r02                    \
        -ortvec mot_deriv.r01.1D mot_deriv_r01                      \
        -ortvec mot_deriv.r02.1D mot_deriv_r02                      \
        -polort 2                                                   \
        -num_stimts 0                                               \
        -jobs 8                                                     \
        -fout -tout -x1D X.xmat.1D -xjpeg X.jpg                     \
        -x1D_uncensored X.nocensor.xmat.1D                          \
        -fitts fitts.$subj.$hemi.niml.dset                          \
        -errts errts.${subj}.$hemi.niml.dset                        \
        -x1D_stop                                                   \
        -bucket stats.$subj.$hemi.niml.dset

So I think the regressors you are looking for are called:

ROI.WMe_rall.1D 
ROI.CSFe_rall.1D

... which do exist in the results directory. Any other regressors you want should be named there, as well.

--pt

1 Like
5

I think lh/rh problem is due to a recent change that I made for group analysis. Let me take a closer look...

-rick

1 Like
6
  1. I tried what you asked for me.

  • AFNI_Data6/FT_analysis: Your script worked without any error for the given dataset.
    Additionally, I tried my script with std.60.sub-100307_?h.spec for my dataset. It worked, but it did not work for sub-100307_?h.spec with the same error I posted before.

  • afni_system_check.py -check_all:
    For the summary, it gave below

================ summary, please fix: ===============

  • just be aware : login shell 'bash', but our code examples use 'tcsh'
  • 'afni' executable is owned by root
  • missing R packages (see rPkgsInstall)
  • dot file test : want 4 modifications across 3 files:
  • insufficient data for AFNI bootcamp
    (see "Prepare for Bootcamp" on install pages)
  1. Regressor
    I found where they are. Thanks. I wanted to know that because I would like to apply the other option for 3dVol2Surf.
7

Yes, the 'sub' prefix triggered a change for subject ID detection. I am putting in a fix for that and we will presumably build new binaries tonight.
Do you build your own binaries (the short system check output does not include it)?

Thanks,
-rick

1 Like
8

image
image1632×908 176 KB

I did not know about the binaries on my server. However, when I ran "afni_system_check.py -check_all", I found the above massage.
We actually did have a problem related to the binaries "libgsl.so.19". Since that was our of my knowledge, I requested our IT tech to resolve it. So, I don't know anything. Sorry.

Thanks,
MJ

9

This is helpful, thanks. Though it does not contain the line I was looking for, the gsl problem actually clears that aspect up.
So I will look into making an update so that you do not need to worry about libgsl, too, and let you know if it gets in.

Thanks,
-rick

1 Like
10

-------------------------------- general ---------------------------------
architecture: 64bit ELF
cpu type: x86_64
system: Linux
release: 5.4.0-205-generic
version: #225-Ubuntu SMP Fri Jan 10 22:23:35 UTC 2025
distribution: Ubuntu 20.04.6 LTS
number of CPUs: 160
apparent login shell: bash
shell RC file: .bashrc (exists)

--------------------- AFNI and related program tests ---------------------
which afni : /usr/share/afni-current/abin/afni
afni version : Precompiled binary linux_ubuntu_16_64: Oct 31 2024
: AFNI_24.3.06 'Elagabalus'
AFNI_version.txt : AFNI_24.3.06, linux_ubuntu_16_64, Oct 31 2024, official
which python : /home/mjeong32@ad.wisc.edu/miniconda/bin/python
python version : 3.11.4
which R : /usr/bin/R
R version : R version 3.6.3 (x86_64-pc-linux-gnu)

instances of various programs found in PATH:
afni : 1 (/usr/share/AFNI_24.3.06/abin/afni)
R : 1 (/usr/bin/R)
python : 2
/home/mjeong32@ad.wisc.edu/miniconda/bin/python3.11
/usr/bin/python3.8
python2 : 1 (/usr/bin/python2.7)
python3 : 2
/home/mjeong32@ad.wisc.edu/miniconda/bin/python3.11
/usr/bin/python3.8

testing ability to start various programs...
afni : success
suma : success
3dSkullStrip : success
3dAllineate : success
3dRSFC : success
SurfMesh : success
3dClustSim : success
uber_subject.py : success
3dMVM : success

------------------------ dependent program tests -------------------------
checking for dependent programs...

which tcsh : /usr/bin/tcsh
tcsh version : 6.21.00
which Xvfb : /usr/bin/Xvfb

checking for R packages...
rPkgsInstall -pkgs ALL -check : FAILURE

    oo Warning: 
       These packages are not installed on the computer: afex!
    These packages are not installed on the computer: phia!
    These packages are not installed on the computer: lmerTest!
    These packages are not installed on the computer: brms!
    These packages are not installed on the computer: metafor!

R RHOME : /usr/lib/R

------------------------------ python libs -------------------------------

++ module loaded: matplotlib.pyplot
module file : /home/mjeong32@ad.wisc.edu/miniconda/lib/python3.11/site-packages/matplotlib/pyplot.py
matplotlib version : 3.10.0

++ module loaded: flask
module file : /home/mjeong32@ad.wisc.edu/miniconda/lib/python3.11/site-packages/flask/init.py
flask version : 3.1.0

++ module loaded: flask_cors
module file : /home/mjeong32@ad.wisc.edu/miniconda/lib/python3.11/site-packages/flask_cors/init.py
flask_cors version : 5.0.0

-------------------------------- env vars --------------------------------
PATH = /home/mjeong32@ad.wisc.edu/miniconda/bin:/home/mjeong32@ad.wisc.edu/miniconda/condabin:~/miniconda3/bin:/home/mjeong32@ad.wisc.edu/miniconda/bin:/usr/local/freesurfer-current/bin:/usr/local/freesurfer-current/fsfast/bin:/usr/local/freesurfer-current/tktools:/usr/local/freesurfer-current/mni/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/share/afni-current/abin:/usr/local/share/apptainer/bin

PYTHONPATH =
R_LIBS =
LD_LIBRARY_PATH =
DYLD_LIBRARY_PATH =
DYLD_FALLBACK_LIBRARY_PATH =
CONDA_SHLVL = 1
CONDA_DEFAULT_ENV = base
CC =

----------------------------- eval dot files -----------------------------

----------- AFNI $HOME files -----------

.afnirc                   : found
.sumarc                   : found
.afni/help/all_progs.COMP : found

--------- shell startup files ----------

-- no .tcshrc, will create one as a follower of .cshrc
-- considered operations: path, apsearch

-- note: followers should not need edits, so edit flags should be 0
(have 1 follower(s), which can be ignored)

dot file test : want 4 modifications across 3 files:

  file             path  flatdir  apsearch        follower
  ---------------  ----  -------  --------        --------
  .cshrc           1     0        1               0     
  .tcshrc          0     0        0               1     
  .bashrc          1     0        1               0

------------------------------ data checks -------------------------------
data dir : found AFNI_data6 under $HOME (5679851M Avail)
top history: 22 Jan 2025 [rickr]: add AV2_aud_AM2.txt
data dir : missing AFNI_demos
data dir : missing suma_demo
data dir : missing afni_handouts
atlas : found TT_N27+tlrc under /usr/share/afni-current/abin

------------------------------ OS specific -------------------------------
which apt-get : /usr/bin/apt-get
apt-get version : apt 2.0.10 (amd64)

have Ubuntu system: Ubuntu 20.04.6 LTS
have Ubuntu afni : Precompiled binary linux_ubuntu_16_64: Oct 31 2024

========================= summary, please fix: =========================

  • just be aware: login shell 'bash', but our code examples use 'tcsh'
  • 'afni' executable is owned by root
  • missing R packages (see rPkgsInstall)
  • dot file test : want 4 modifications across 3 files:
  • insufficient data for AFNI bootcamp
    (see "Prepare for Bootcamp" on install pages)

Oh, sorry.
Just in case, I posted all output from the command "afni_system_check.py -check all"

I really appreciate your help.

Thanks,
MJ

11

Hi MJ,

Finally starting the build, it should be ready in the morning.
After updating, see if there is still a problem with the libgsl version, hopefully not.

Thanks,

-rick

1 Like
12

It turns out the gsl libraries did not make it into the distribution. There is a more subtle place that our build scripts need editing to grab that, sorry. I will try to include the library with the next build.
Anyway, at least the main code fix should be available now.
-rick

1 Like