@ss_review_html Error

AFNI version info (afni -ver): Feb 5 2024 (Version AFNI_24.0.05 'Caracalla')

Hi AFNI team,

I'm running into a strange issue when running my subject proc script after updating to the latest AFNI version. My script runs fine until this point:

gen_epi_review.py -script @epi_review.AD1173 -dsets pb00.AD1173.r01.tcat+orig.HEAD pb00.AD1173.r02.tcat+orig.HEAD pb00.AD1173.r03.tcat+orig.HEAD pb00.AD1173.r04.tcat+orig.HEAD
cat
cat out.ap_uvars.txt
afni_python_wrapper.py -eval data_file_to_json()
gen_ss_review_scripts.py -exit0 -init_uvars_json out.ap_uvars.json -write_uvars_json out.ss_review_uvars.json
** SVWD: invalid <noname> variable: tlrc_base
++ writing ss review basic:          @ss_review_basic
++ writing ss review driver:         @ss_review_driver
++ writing ss review drive commands: @ss_review_driver_commands

After that, it runs fine again until here:

apqc_make_tcsh.py -review_style pythonic -subj_dir . -uvar_json out.ss_review_uvars.json
++ Done making (executable) IC errts script: 
      run_instacorr_errts.tcsh
++ Done making (executable) GV errts script: 
      run_graphview_errts.tcsh
++ Done making (executable) IC pbrun script: 
      run_instacorr_pbrun.tcsh
++ Done making (executable) GV pbrun script: 
      run_graphview_pbrun.tcsh
++ APQC create: qc_00_vorig_EPI
++ APQC create: qc_01_vorig_anat
++ APQC create: qc_02_vorig_olap
++ APQC create: qc_03_ve2a_epi2anat
++ APQC create: qc_04_va2t_anat2temp
++ APQC create: qc_05_va2t_mask2final
++ Found 4 stats items to use for vstat QC block
++ APQC create: qc_06_vstat_Full_Fstat
++ APQC create: qc_07_vstat_Choice_Fstat
++ APQC create: qc_08_vstat_AttentionCheck_Fstat
++ APQC create: qc_09_vstat_NoResponse_0_Coef
++ APQC create: qc_10_mot_enormoutlr
++ APQC create: qc_11_mot_VR6
++ APQC create: qc_12_mot_grayplot
++ APQC create: qc_13_regr_ideal
++ APQC create: qc_14_regr_stims
++ APQC create: qc_15_regr_df
++ APQC create: qc_16_regr_corr_errts
++ APQC create: qc_17_regr_tsnr_vreg
++ pb for volreg is: pb02
++ APQC create: qc_18_regr_tsnr_fin
++ APQC create: qc_19_radcor_rc_tcat
                run: r01/4
                run: r02/4
                run: r03/4
                run: r04/4
++ APQC create: qc_20_radcor_rc_volreg
                run: r01/4
                run: r02/4
                run: r03/4
                run: r04/4
++ APQC create: qc_21_warns_xmat
++ APQC create: qc_22_warns_cen_total
++ APQC create: qc_23_warns_cen_stim
++ APQC create: qc_24_warns_press
++ APQC create: qc_25_warns_flip
++ APQC create: qc_26_qsumm_ssrev
++ APQC create: copy jsons to info dir
++ APQC create: copy ss_review_basic file
++ APQC create: display ss_review_basic info

subject ID                : AD1173
AFNI version              : AFNI_24.0.05
AFNI package              : linux_ubuntu_16_64
TR                        : 2.0
TRs removed (per run)     : 0
multiband level           : 1
slice timing pattern      : alt+z2
num stim classes provided : 3
final anatomy dset        : anat_final.AD1173+tlrc.HEAD
final stats dset          : stats.AD1173_REML+tlrc.HEAD
orig voxel counts         : 64	64	36
orig voxel resolution     : 3.437500	3.437500	3.400000
orig volume center        : -3.134750	-20.717148	50.453110
final voxel resolution    : 3.000000	3.000000	3.000000

motion limit              : 1.0
num TRs above mot limit   : 1
average motion (per TR)   : 0.105914
average censored motion   : 0.0987852
max motion displacement   : 2.3118
max censored displacement : 2.3118
outlier limit             : 0.1
average outlier frac (TR) : 0.00503524
num TRs above out limit   : 4
flip guess                : NO_FLIP

num runs found            : 4
num TRs per run           : 128 128 128 128
num TRs per run (applied) : 128 128 122 128
num TRs per run (censored): 0 0 6 0
fraction censored per run : 0 0 0.046875 0
TRs total (uncensored)    : 512
TRs total                 : 506
degrees of freedom used   : 43
degrees of freedom left   : 463
final DF fraction         : 0.904297

TRs censored              : 6
censor fraction           : 0.011719
num regs of interest      : 7
num TRs per stim (orig)   : 455 455 455 32 32 32 44
num TRs censored per stim : 6 6 6 0 0 0 0
fraction TRs censored     : 0.013 0.013 0.013 0.000 0.000 0.000 0.000
ave mot per sresp (orig)  : 0.105067 0.105067 0.105067 0.105288 0.105288 0.105288 0.096733
ave mot per sresp (cens)  : 0.097086 0.097086 0.097086 0.105288 0.105288 0.105288 0.096733

TSNR average              : 238.653
global correlation (GCOR) : 0.0618341
anat/EPI mask Dice coef   : 0.857355
anat/templ mask Dice coef : 0.932313
maximum F-stat (masked)   : 33.4029
blur estimates (ACF)      : 0.578951 3.3972 10.6908
blur estimates (FWHM)     : 0 0 0


# ++++++++++++++ Check output of @ss_review_basic ++++++++++++++ #
------------------------------------------------------------------
------------------------------------------------------------------

++ Done setting up QC dir: QC_AD1173
   To create the APQC HTML, run either this (from any location):      
     
       apqc_make_html.py -qc_dir /nfs/mint-results/DD_results_dmBLOCK_2.2.24/subj_results/AD1173.results_dmBLOCK_test/QC_AD1173

   ... or this (from the afni_proc.py results directory):

       apqc_make_html.py -qc_dir QC_AD1173


tcsh @ss_review_html
tee out.review_html
@ss_review_html: No such file or directory.

At this point, the script crashes. I've tried troubleshooting this, but currently stumped so any help is appreciated. Thank you!

Sincerely,
Ian

Hi Ian,

What are the contents of out.ap_uvars.txt?

Thanks,

  • rick

The contents are:

  mot_limit      : 1.0
  out_limit      : 0.1
  copy_anat      : anatSS.AD1173+orig.HEAD
  mask_dset      : mask_epi_anat.AD1173+tlrc.HEAD
  tlrc_base      : MNI152_2009_template_SSW.nii.gz
  ss_review_dset : out.ss_review.AD1173.txt

In case it's relevant, here's the output of my afni_system_check too - of note, is that one of the R packages is no longer installing for me now:

-------------------------------- general ---------------------------------
architecture:         64bit ELF
cpu type:             x86_64
system:               Linux
release:              5.4.0-169-generic
version:              #187-Ubuntu SMP Thu Nov 23 14:52:28 UTC 2023
distribution:         Ubuntu 20.04.6 LTS
number of CPUs:       96
apparent login shell: bash
shell RC file:        .bashrc (exists)

--------------------- AFNI and related program tests ---------------------
which afni           : /nfs/pkg64/afni_ubuntu_16_20240205/afni
afni version         : Precompiled binary linux_ubuntu_16_64: Feb  5 2024 
                     : AFNI_24.0.05 'Caracalla'
AFNI_version.txt     : AFNI_24.0.05, linux_ubuntu_16_64, Feb 05 2024, official
which python         : /nfs/mint/scripts/DD_scripts/matplotlib_venv/bin/python
python version       : 3.8.10
which R              : /bin/R
R version            : R version 4.3.2 (x86_64-pc-linux-gnu)

instances of various programs found in PATH:
    afni    : 1   (/nfs/pkg64/afni_ubuntu_16_20240205/afni)
    R       : 1   (/usr/bin/R)
    python  : 1   (/usr/bin/python3.8)
    python2 : 1   (/usr/bin/python2.7)
    python3 : 1   (/usr/bin/python3.8)


testing ability to start various programs...
    afni                 : success
    suma                 : success
    3dSkullStrip         : success
    3dAllineate          : success
    3dRSFC               : success
    SurfMesh             : success
    3dClustSim           : success
    uber_subject.py      : success
    3dMVM                : success

------------------------ dependent program tests -------------------------
checking for dependent programs...

which tcsh           : /bin/tcsh
which Xvfb           : /bin/Xvfb

checking for R packages...
    rPkgsInstall -pkgs ALL -check : FAILURE
        
        oo Warning: 
           These packages are not installed on the computer: brms!
        

R RHOME : /usr/lib/R

------------------------------ python libs -------------------------------

++ module loaded: matplotlib.pyplot
   module file : /nfs/mint/scripts/DD_scripts/matplotlib_venv/lib/python3.8/site-packages/matplotlib/pyplot.py
   matplotlib version : 3.7.1

** failed to load module flask
-- flask is not required, but is desirable

** failed to load module flask_cors
-- flask_cors is not required, but is desirable

-------------------------------- env vars --------------------------------
PATH = /nfs/mint/scripts/DD_scripts/matplotlib_venv/bin:/nfs/pkg64/afni_ubuntu_16_20240205:/home/farnsworth/bin:/home/farnsworth/.local/bin:/usr/local/cuda/bin:/bin:/usr/bin:/opt/thinlinc/bin:/usr/local/bin:/usr/bin/X11:/sbin:/usr/sbin:/usr/local/sbin:/snap/bin:/nfs/pkg64/afni_ubuntu_16_20240205

PYTHONPATH = 
R_LIBS = /home/farnsworth/R

LD_LIBRARY_PATH = /usr/local/cuda/lib64:/opt/thinlinc/lib64:/opt/thinlinc/lib

DYLD_LIBRARY_PATH = 
DYLD_FALLBACK_LIBRARY_PATH = 

----------------------------- eval dot files -----------------------------

----------- AFNI $HOME files -----------

    .afnirc                   : found
    .sumarc                   : found
    .afni/help/all_progs.COMP : found

--------- shell startup files ----------

   -- good: .tcshrc seems to contain 'source .cshrc'
   -- considered operations: path, apsearch
   
   -- note: followers should not need edits, so edit flags should be 0
      (have 1 follower(s), which can be ignored)
   
   dot file test : want 1 modifications across 3 files:
   
      file             path  flatdir  apsearch        follower
      ---------------  ----  -------  --------        --------
      .cshrc           1     0        0               0     
      .tcshrc          0     0        0               1     
      .bashrc          0     0        0               0     
   
------------------------------ data checks -------------------------------

data dir : missing AFNI_data6
data dir : missing AFNI_demos
data dir : missing suma_demo
data dir : missing afni_handouts
atlas    : found TT_N27+tlrc  under /nfs/pkg64/afni_ubuntu_16_20240205

------------------------------ OS specific -------------------------------
which apt-get        : /bin/apt-get
apt-get version      : apt 2.0.10 (amd64)

have Ubuntu system: Ubuntu 20.04.6 LTS
have Ubuntu afni  : Precompiled binary linux_ubuntu_16_64: Feb  5 2024 

=========================  summary, please fix:  =========================
*  just be aware: login shell 'bash', but our code examples use 'tcsh'
*  'afni' executable is owned by root
*  missing R packages (see rPkgsInstall)
*  dot file test : want 1 modifications across 3 files:
*  insufficient data for AFNI bootcamp
   (see "Prepare for Bootcamp" on install pages)

Howdy-

I'm a little confused to see this after the apqc_make_html.py command:

tcsh @ss_review_html
tee out.review_html
@ss_review_html: No such file or directory.

That shouldn't actually appear anymore. It has been a long time since that style of making the QC HTML was done.

Is it possible you have multiple versions of AFNI on your computer, or that you are rerunning the APQC HTML on old afni_proc.py results?

--pt

This is because the proc script was generated with a much older version of afni_proc.py, but is being run on a newer system.

At this point, it would be much better to rerun the same afni_proc.py command and the resulting proc script. While afni_proc.py adds checks to be sure that a new version of a proc script is not run on an old system, it is harder to check that an old proc script is being run on a newer system.

You should re-run afni_proc.py to get a processing script that matches your version.

  • rick

Hi Paul and Rick,

Oops, you're both right! I completely forgot to update the afni_proc script after updating the AFNI version (I was testing on an existing subject's proc script from over a year ago). I generated an updated afni_proc script and now it's smooth sailing! Apologies for the simple mistake, and thank you for the help.

Sincerely,
Ian

Hi, Ian-

Not a problem at all, glad that sorted out the mystery.

-pt