QC directory not created on a centos system

hi guys

i'm working on a centos cluster that had afni built in a somewhat piecemeal way but it's been okay until now. we've run @SUMA_Make_Spec and parts of afni_procs before without issue. however recently we began running a full normal afni_proc and we've noticed the QC_* directories are not created. this happens whether "pythonic" or "basic" is specified to -html_review_style.

the apqc_make_tcsh.py and apqc_make_html.py programs both appear to be installed correctly, but i did notice some weirdness in the out.ss_review.sub001.txt file; that file contains all the normal stuff at first, up through line 37 ("final DF fraction") but then line 39 says:
rc: Subscript out of range.

any ideas? let me know if you need any more files or info and THANKS. here's the afni_system_check.py output for the system:

-------------------------------- general ---------------------------------
architecture: 64bit ELF
cpu type: x86_64
system: Linux
release: 3.10.0-1160.105.1.el7.x86_64
version: #1 SMP Thu Dec 7 15:39:45 UTC 2023
distribution: CentOS Linux 7.9.2009 Core
number of CPUs: 4
apparent login shell: bash
shell RC file: .bashrc (exists)

--------------------- AFNI and related program tests ---------------------
which afni : /opt/afni/abin/afni
afni version : Precompiled binary linux_centos_7_64: Sep 7 2023
: AFNI_23.2.08 'Marcus Didius Severus Julianus'
AFNI_version.txt : AFNI_23.2.08, linux_centos_7_64, Sep 07 2023, local
which python : /usr/bin/python
python version : 3.6.8
which R : /opt/R/bin/R
R version :

instances of various programs found in PATH:
afni : 1 (/opt/afni/abin/afni)
R : 2
/opt/R/r-wrapper.sh
/usr/bin/R
python : 2
/usr/bin/python2.7
/usr/bin/python3.6
python2 : 1 (/usr/bin/python2.7)
python3 : 1 (/usr/bin/python3.6)

testing ability to start various programs...
afni : success
suma : success
3dSkullStrip : success
3dAllineate : success
3dRSFC : success
SurfMesh : success
3dClustSim : success
uber_subject.py : success
3dMVM : success

------------------------ dependent program tests -------------------------
checking for dependent programs...

which tcsh : /usr/bin/tcsh
which Xvfb : /usr/bin/Xvfb

checking for R packages...
rPkgsInstall -pkgs ALL -check : FAILURE
/mnt/c7_opt/afni/abin/@global_parse: Command not found.

R RHOME : You must set the RVERSION environment variable before using this command.

export RVERSION=N

Available versions:

3.6.0
4.2.1
4.3.1
4.3.2

------------------------------ python libs -------------------------------

++ module loaded: matplotlib.pyplot
module file : /opt/afni/AFNI_23.2.08/venv/lib64/python3.6/site-packages/matplotlib/pyplot.py
matplotlib version : 3.3.4

-------------------------------- env vars --------------------------------
PATH = /opt/grid7/sge/bin:/opt/grid7/sge/bin/lx-amd64:/opt/mico/mico-build/:/opt/muse/muse-3.0.5/bin/:/opt/mass/mass-1.1.1/bin/:/opt/dramms/1.5.1/bin/:/opt/SPM/bin:/usr/local/bin/slicer:/opt/grid7/sge/bin:/opt/grid7/sge/bin/lx-amd64:/opt/qit/qit-build-linux-latest/bin:/opt/nodejs/v12.18.3/bin:/home/vhasfctorris/.npm-packages/bin:/opt/mipav/mipav:/opt/mathworks/bin:/opt/itk-snap:/opt/grid_tools/cnt_qstat/build/bin:/opt/fsl/bin:/opt/freesurfer/bin:/opt/fmriprep/bin:/opt/dcmtk/3.6.0/bin:/opt/ants/bin:/opt/afni/abin:/opt/R/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/local/bin/mricron:/opt/pijp-fs/bin:/opt/pijp-nnicv/bin:/opt/pijp-dti/bin:/opt/pijp-asl/bin:/opt/pijp-ants/bin:/opt/pijp-fwe/bin:/opt/pijp-matasl/bin:/opt/pijp-mriqc/bin:/opt/pijp-multiatlas/bin:/opt/pijp-spmseg/bin:/opt/pijp-oxasl/bin:/opt/pijp-asl-mot/bin:/opt/pijp-wmls/bin:/opt/pijp-export/bin:/opt/pycharm/pycharm-community-2022.1.3/bin:/opt/python/3.10.5/bin:/opt/python/3.9.13/bin:/home/vhasfctorris/.local/bin:/home/vhasfctorris/bin

PYTHONPATH =
R_LIBS =
LD_LIBRARY_PATH =
DYLD_LIBRARY_PATH =
DYLD_FALLBACK_LIBRARY_PATH =

----------------------------- eval dot files -----------------------------

----------- AFNI $HOME files -----------

.afnirc                   : found
.sumarc                   : found
.afni/help/all_progs.COMP : found

--------- shell startup files ----------

-- no .tcshrc, will create one as a follower of .cshrc
-- considered operations: path, flatdir, apsearch
(not on a mac, should skip flatdir)

-- note: followers should not need edits, so edit flags should be 0
(have 1 follower(s), which can be ignored)

dot file test : want 4 modifications across 3 files:

  file             path  flatdir  apsearch        follower
  ---------------  ----  -------  --------        --------
  .cshrc           1     0        1               0     
  .tcshrc          0     0        0               1     
  .bashrc          1     0        1               0     

------------------------------ data checks -------------------------------
data dir : missing AFNI_data6
data dir : missing AFNI_demos
data dir : missing suma_demo
data dir : missing afni_handouts
atlas : found TT_N27+tlrc under /opt/afni/abin

------------------------------ OS specific -------------------------------
which yum : /usr/bin/yum
yum version : 3.4.3

========================= summary, please fix: =========================

  • just be aware: login shell 'bash', but our code examples use 'tcsh'
  • 'afni' executable is owned by root
  • missing R packages (see rPkgsInstall)
  • dot file test : want 4 modifications across 3 files:
  • insufficient data for AFNI bootcamp
    (see "Prepare for Bootcamp" on install pages)

Ciao, Sam-

Hm, not an ideal situation, but we can investigate. Probably not good that "root" owns afni...

Seeing the output.proc.* would help greatly to try to see what is happening, for a start.

Now or later, you could try running:

apqc_make_tcsh.py -review_style basic -subj_dir . \
    -uvar_json out.ss_review_uvars.json \
    -do_log 

and that will create a "log" file in the results directory, or error out in some way.

And that computer's ~/.tcshrc file contain source ~/.cshrc?

--pt

Ciao, Paul-

yeah, i didn't / couldn't install AFNI on this system, sigh

thank you so much and ok... i think we're getting closer. i hadn't run that job but my colleague had. he may have different settings in his rc files or whatnot (solution may lie there). but in the meantime i'll tell you what i just did:

i logged into this system through my portal (the only one i can) and just manually ran apqc_make_tcsh.py as you suggested in the .results directory (without "-do_log" because that option didn't exist yet in our installed version) and it ran! it created a QC_ directory. then i ran apqc_make_html.py and that worked too and its outputs look normal. note that i don't even have a .tcshrc in my home... i have a .bashrc and a .cshrc. what should we look for in them? any idea how discrepancies with my home dir .*rc files and his may make the difference? here's the beginning and ending output of his run of the afni_proc (the run where the QC dir wasn't created... and only beginning and ending because it wouldn't let me paste the whole thing):

echo auto-generated by afni_proc.py, Thu May 16 14:52:26 2024
auto-generated by afni_proc.py, Thu May 16 14:52:26 2024
echo (version 7.60, August 21, 2023)
(version 7.60, August 21, 2023)
echo execution started: date
date
execution started: Thu May 16 14:52:44 PDT 2024
afni -ver
Precompiled binary linux_centos_7_64: Sep 7 2023 (Version AFNI_23.2.08 'Marcus Didius Severus Julianus')
afni_history -check_date 14 Nov 2022
-- is current: afni_history as new as: 14 Nov 2022
most recent entry is: 01 Sep 2023

...

-- Wrote edited dataset: ./sphere.mean.20.d02+tlrc.BRIK

++ 3dTcorrelate: AFNI version=AFNI_23.2.08 (Sep 7 2023) [64-bit]
++ Wrote dataset: ./radcor.20.d02.errts.corr+tlrc.BRIK
++ 3dbucket: AFNI version=AFNI_23.2.08 (Sep 7 2023) [64-bit]
---- done

++ have do_clean, cleaning up...

gen_epi_review.py -script @epi_review.sub001_DPOP_resting_state -dsets pb00.sub001_DPOP_resting_state.r01.tcat+orig.HEAD
cat
cat out.ap_uvars.txt
afni_python_wrapper.py -eval data_file_to_json()
gen_ss_review_scripts.py -exit0 -init_uvars_json out.ap_uvars.json -write_uvars_json out.ss_review_uvars.json
** failed to guess at any stats dset, resting state?
(else X-matrix file "X.xmat.1D" may not apply)
++ writing ss review basic: @ss_review_basic
** no stats_dset dset, cannot drive view_stats, skipping...
++ writing ss review driver: @ss_review_driver
++ writing ss review drive commands: @ss_review_driver_commands
rm -fr rm.all_runs.volreg.mask+tlrc.BRIK.gz rm.all_runs.volreg.mask+tlrc.HEAD rm.all_runs.volreg+tlrc.BRIK rm.all_runs.volreg+tlrc.HEAD rm.censor.r01.1D rm.det_pcin_r01+tlrc.BRIK rm.det_pcin_r01+tlrc.HEAD rm.det_pcin_rall+tlrc.BRIK rm.det_pcin_rall+tlrc.HEAD rm.epi.1.r01+tlrc.BRIK.gz rm.epi.1.r01+tlrc.HEAD rm.epi.all1+orig.BRIK.gz rm.epi.all1+orig.HEAD rm.epi.min.r01+tlrc.BRIK.gz rm.epi.min.r01+tlrc.HEAD rm.epi.nomask.r01+tlrc.BRIK rm.epi.nomask.r01+tlrc.HEAD rm.epi.volreg.r01+orig.BRIK rm.epi.volreg.r01+orig.HEAD rm.errts.unit+tlrc.BRIK rm.errts.unit+tlrc.HEAD rm.mask.anaticor.float+tlrc.BRIK.gz rm.mask.anaticor.float+tlrc.HEAD rm.mask_CSFe+tlrc.BRIK.gz rm.mask_CSFe+tlrc.HEAD rm.mask_CSF+tlrc.BRIK.gz rm.mask_CSF+tlrc.HEAD rm.mask_GMe+tlrc.BRIK.gz rm.mask_GMe+tlrc.HEAD rm.mask_GM+tlrc.BRIK.gz rm.mask_GM+tlrc.HEAD rm.mask_r01+tlrc.BRIK.gz rm.mask_r01+tlrc.HEAD rm.mask_WMe+tlrc.BRIK.gz rm.mask_WMe+tlrc.HEAD rm.mask_WM+tlrc.BRIK.gz rm.mask_WM+tlrc.HEAD rm.mean_r01+tlrc.BRIK rm.mean_r01+tlrc.HEAD rm.noise.all+tlrc.BRIK rm.noise.all+tlrc.HEAD rm.noise.det+tlrc.BRIK rm.noise.det+tlrc.HEAD rm.noise.vreg.r01+tlrc.BRIK rm.noise.vreg.r01+tlrc.HEAD rm.out.cen.r01.1D rm.resam.anat+tlrc.BRIK rm.resam.anat+tlrc.HEAD rm.resam.group+tlrc.BRIK rm.resam.group+tlrc.HEAD rm.ROIPC.vent00.1D rm.ROIPC.vent01.1D rm.ROIPC.vent02.1D rm.ROIPC.vent_eig.1D rm.ROIPC.vent+tlrc.BRIK.gz rm.ROIPC.vent+tlrc.HEAD rm.ROIPC.vent_vec.1D rm.signal.all+tlrc.BRIK rm.signal.all+tlrc.HEAD rm.signal.vreg.r01+tlrc.BRIK rm.signal.vreg.r01+tlrc.HEAD Segsy
if ( -e @ss_review_basic ) then
tee out.ss_review.sub001_DPOP_resting_state.txt
./@ss_review_basic

subject ID : sub001_DPOP_resting_state
AFNI version : AFNI_23.2.08
AFNI package : linux_centos_7_64
TR : 0.82
TRs removed (per run) : 3
multiband level : 1
slice timing pattern : simult
num stim classes provided : 0
final anatomy dset : anat_final.sub001_DPOP_resting_state+tlrc.HEAD
final stats dset : NO_STATS
final errts dset : errts.sub001_DPOP_resting_state.fanaticor+tlrc.HEAD
orig voxel counts : 104 104 72
orig voxel resolution : 2.000000 2.000000 2.000000
orig volume center : -2.204803 -32.817867 -49.509598
final voxel resolution : 2.000000 2.000000 2.000000

motion limit : 0.5
num TRs above mot limit : 0
average motion (per TR) : 0.117086
average censored motion : 0.117086
max motion displacement : 0.475367
max censored displacement : 0.475367
outlier limit : 0.05
average outlier frac (TR) : 0.000233829
num TRs above out limit : 0

num runs found : 1
num TRs per run : 363
num TRs per run (applied) : 363
num TRs per run (censored): 0
fraction censored per run : 0
TRs total (uncensored) : 363
TRs total : 363
degrees of freedom used : 18
degrees of freedom left : 345
final DF fraction : 0.950413

rc: Subscript out of range.

Hi, Sam-

OK. I don't think not having a ~/.tcshrc should be a problem; the system should just fallback to ~/.cshrc.

If you run:

tcsh -x @ss_review_basic

in that results directory, it might provide more information about the exact place that this failing. However, from the error message, I think it is choking at this point in the script (line 5 here):

# ----------------------------------------------------------------------
# report TR counts per stim, and possibly censor counts

# note number of regressors of interest
set rc = ( `1d_tool.py -infile $xstim -show_rows_cols -verb 0` )
set nint = $rc[2]
@ nm1 = $nint - 1

So, that suggests something went bad with the Xmatrix calc (earlier, this is the definition of the xstim var: set xstim = X.stim.xmat.1D). So, why that causes a problem is more the root issue.

Not having the afni_proc.py command, nor the output.proc* text, and knowing that the system has installation quirks are all difficult factors here... For the first two, perhaps you could email me (and possibly Rick) those?

thanks,
pt

ok just messaged you and rick...

Hi, Sam-

Thanks for sending that. I don't see any earlier fails, but searching for that particular error message:

"rc: Subscript out of range" afni

… (which I should have done to start…), brings up this MB post, with the solution that AFNI ver=23.0.09 had a bug in it related to the Xmat processing (that was fortunately soon squashed). Now, your version is 23.0.08, but that seems suspiciously coincidental to see the same error message

So, I suspect that this will be resolved by updating the local binaries version. Please let us know how that goes.

--pt

hi Paul-

we were able to update AFNI on this system, and yes, now the problem is fixed.
THANKS SO MUCH

-Sam

1 Like