SSwarper failed in step 5a when run on new system

AFNI version info (afni -ver): AFNI_21.3.04 (Oct 20 2021) [64-bit]

For some reason, when I run SSwarper v2.53 on our new supercomputer access system, I keep geting errors stating that it failed in step 5a. The program began to run but the only image files created were the anatU, anatUA, and anatUAC files. The output file has this line several times: "ERROR: Xvfb -- not found in path -- program fails". Has anyone come across this error? And if so, how did you resolve it?

Thank you,
Katie

Below is the code for the script I ran for SSWarper:

#!/bin/bash
#SBATCH --nodes=2
#SBATCH --ntasks=8
#SBATCH --time=6:00:00
#SBATCH --mem=32GB
#SBATCH --partition=open


# Get started
echo "Job started on 'hostname' at 'date'"

# Go to correct place
cd $SLURM_SUBMIT_DIR

# Run the job ## CHANGE NAME OF ANATOMICAL IMAGE ##
# **change date in .nii file & update subject id ** 
@SSwarper -input __T1_mpr_ns_sag_ADNI_176mm_20220616101848_2.nii -subid sub884 -base MNI152_2009_template_SSW.nii.gz 


# Finish up
echo "Job ended at 'date'"

Just as a clarification, the anatQQ, anatS, and anatSS files were not created when I ran the above script.

Hi, Katie-

Hm, OK. What is the output of the system check:

afni_system_check.py -check_all

? Maybe there is a dependency missing.

thanks,
pt

Hi Pt,

Thank you very much for the suggestion! The output for that system check is long. Towards the end it says "missing R packages"--maybe that is the issue? Should I go ahead and run the 3 suggestions following that line? Here is the entire output below:

[kxm5964@p-sc-2329 MID]$ afni_system_check.py -check_all
-------------------------------- general ---------------------------------
architecture:         64bit 
system:               Linux
release:              4.18.0-425.19.2.el8_7.x86_64
version:              #1 SMP Fri Mar 17 01:52:38 EDT 2023
distribution:         Red Hat Enterprise Linux 8.7 Ootpa
number of CPUs:       48
apparent login shell: bash
shell RC file:        .bashrc (exists)

--------------------- AFNI and related program tests ---------------------
which afni           : /swst/apps/afni/21.3.04_gcc-8.5.0/bin/afni
afni version         : Precompiled binary linux_centos_7_64: Oct 20 2021 
                     : AFNI_21.3.04 'Trajan'
AFNI_version.txt     : AFNI_21.3.04, linux_centos_7_64, Oct 20 2021
which python         : /usr/bin/python
python version       : 3.6.8
which R              : /usr/bin/which: no R in (/swst/apps/afni/21.3.04_gcc-8.5.0/bin:/swst/apps/afni/21.3.04_gcc-8.5.0:/opt/TurboVNC/bin:/storage/home/kxm5964/.local/bin:/storage/home/kxm5964/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/storage/icds/tools/bin:/storage/sys/slurm/bin)

which tcsh           : /usr/bin/tcsh

instances of various programs found in PATH:
    afni    : 1   (/storage/icds/swst/deployed/production/20220813/apps/afni/21.3.04_gcc-8.5.0/bin/afni)
    R       : 0 
    python  : 1   (/usr/libexec/platform-python3.6)
    python2 : 1   (/usr/bin/python2.7)
    python3 : 1   (/usr/libexec/platform-python3.6)


testing ability to start various programs...
    afni                 : success
    suma                 : FAILURE
        suma: error while loading shared libraries: libGLw.so.1: cannot open shared object file: No such file or directory
    3dSkullStrip         : FAILURE
        3dSkullStrip: error while loading shared libraries: libGLw.so.1: cannot open shared object file: No such file or directory
    uber_subject.py      : success
    3dAllineate          : success
    3dRSFC               : FAILURE
        3dRSFC: error while loading shared libraries: libgsl.so.0: cannot open shared object file: No such file or directory
    SurfMesh             : FAILURE
        SurfMesh: error while loading shared libraries: libGLw.so.1: cannot open shared object file: No such file or directory
    3dClustSim           : success
    3dMVM                : success

checking for R packages...
    rPkgsInstall -pkgs ALL -check : FAILURE
        R: Command not found.

R RHOME : R: Command not found.

checking for $HOME files...
    .afnirc                   : missing
    .sumarc                   : missing
    .afni/help/all_progs.COMP : missing

------------------------------ python libs -------------------------------
** failed to load module PyQt4
-- PyQt4 is no longer needed for an AFNI bootcamp

** failed to load module matplotlib.pyplot
-- matplotlib.pyplot is not required, but is desirable

-------------------------------- env vars --------------------------------
PATH = /swst/apps/afni/21.3.04_gcc-8.5.0/bin:/swst/apps/afni/21.3.04_gcc-8.5.0:/opt/TurboVNC/bin:/storage/home/kxm5964/.local/bin:/storage/home/kxm5964/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/storage/icds/tools/bin:/storage/sys/slurm/bin

PYTHONPATH = 
R_LIBS = 
LD_LIBRARY_PATH = 
DYLD_LIBRARY_PATH = 
DYLD_FALLBACK_LIBRARY_PATH = 

------------------------------ data checks -------------------------------
data dir : missing AFNI_data6
data dir : missing AFNI_demos
data dir : missing suma_demo
data dir : missing afni_handouts
atlas    : found TT_N27+tlrc  under /swst/apps/afni/21.3.04_gcc-8.5.0/bin

------------------------------ OS specific -------------------------------
which dnf            : /usr/bin/dnf
dnf version          : 4.7.0
which yum            : /usr/bin/yum
yum version          : 4.7.0


=========================  summary, please fix:  =========================
*  just be aware: login shell 'bash', but our code examples use 'tcsh'
*  failure under initial "AFNI and related program tests"
*  AFNI programs show FAILURE
*  missing R packages (see rPkgsInstall)
*  please run: cp /swst/apps/afni/21.3.04_gcc-8.5.0/bin/AFNI.afnirc ~/.afnirc
*  please run: "suma -update_env" for .sumarc
*  please run: apsearch -update_all_afni_help
*  insufficient data for AFNI bootcamp

Hi, Katie-

There are a few points here:

  • There appear to be some missing dependencies/libraries, as noted in the "testing ability to start various programs..." section. These include libGLw.so.1 and libgsl.so.0. I think this is the root of the program failure, likely. I'm a little surprised there weren't earlier program failures, actually, since 3dSkullStrip is one of the failing programs.

  • Your operating system is RedHat 8, and we have had build descriptions for earlier versions of Red Hat, through RHel 7, but now we have a build for "RedHat/CentOS/Rocky 8 Linux"; we have to add this to our list online, but this text file contains the underlying dependencies:
    https://github.com/afni/afni/blob/master/src/other_builds/OS_notes.linux_rocky_8.txt
    ... and then you should be able to install the "linux_rock_8.tgz" package. Is that feasible? Is this operating system one you have admin rights on, or is it managed by someone else?

  • The missing R packages aren't the origin of the current problem---@SSwarper doesn't use them, but you might want to install those at some point, for the purpose of group analysis (many AFNI group analysis programs are written in R and use their libraries).

  • Yes, the three "Please fix" items would be good to do; they don't require admin privileges, so you can copy+paste these as suggested:

    *  please run: cp /swst/apps/afni/21.3.04_gcc-8.5.0/bin/AFNI.afnirc ~/.afnirc
    *  please run: "suma -update_env" for .sumarc
    *  please run: apsearch -update_all_afni_help
    
  • Your system's version of AFNI is relatively old now (a couple years), so updating the binaries will also get you updated code, too, which might be useful.

--pt

Hi pt,

Thank you very much for the help! I really appreciate it. Unfortunately I do not have admin priviledges for this system, but I have put in an IT request and hopefully they will be able to address this soon. When I hear back from them I'll let you know if that ends up fixing the problem.

Thank you again!
Katie

Hi pt,

I tried applying as many of your suggestions as possible with the help of our IT team. The IT person I was working with suggested that I load the gsl module to fix the gsl errors. I did this, but I'm not sure if it fully addresses your first bullet point suggestion. Should I also load a "GL" library prior to running the script? The IT member wasn't sure about the OpenGL library, but advised running the job as an interactive node type instead of using standard cores. Neither of these changes fixed the problem though.

Furthermore, the IT member said that AFNI was installed via a semi-automated software installer system called Spack, so he suspected that the installation is compatible with RHEL 8 OS. However, he said that if we couldn't solve the issue with other means, we could bring in the Operations team to verify the AFNI installation.

I did not install the missing R packages since I'm not sure I have priviledges to do that. I did copy and paste the "Please fix" items, however, and the first and last changes worked fine. However, I got an error when I tried to run "suma -update_env" for .sumarc. It returned "-bash: suma -update_env: command not found". Do you have any suggestions for fixing this?

Thank you again for all your help!
Katie

Hi, Katie-

The two libraries mentioned above that appear to be missing should be installable by making sure that these
RedHat/CentOS/Rocky 8 Linux dependencies have all been installed. Specifically, I think that the libgsl* dependency is met by the gsl-devel package, and the libGLw* one by mesa-libGLw-devel mesa-libGLU-devel. But to be sure, I would run both lines 6-7 there, to make sure nothing else got skipped. That requires admin privileges, so would have to be done by your administrators. I am not familiar with Spack, so I'm not sure about checking on missed dependencies in it---hopefully the admins will be able to work with that aspect. Note that running afni_system_check.py -check_all should tell you whether those dependencies have been met, after the admins have taken a swing at getting those missing items installed.

Ah, I should have realized and noted that suma -update_env from the "Please Fix" list will fail currently, because it has a dependency on one of those currently-missing libraries. You can see it in the system check output:

    suma                 : FAILURE
        suma: error while loading shared libraries: libGLw.so.1: cannot open shared object file: No such file or directory

You will have to wait on running that until your administrators have sorted out the missing libraries. Sorry about that.

--pt

Thank you so much pt! I forgot to let you know, but this was resolved early last month after implementing all the changes you suggested.

Ah, great to know, thanks. Glad that worked.

--pt