Containerized AFNI on Computing Cluster

AFNI Message Board
1

Hi AFNI legends,

Our admin has built an AFNI container for our computing cluster. It seems to work OK, but I get an error with the X11 visualization, in SSWarp, SUMA and in afni_proc.

 -- trying to start Xvfb :680 
[1] 24060
_XSERVTransmkdir: ERROR: euid != 0,directory /tmp/.X11-unix will not be created.
The XKEYBOARD keymap compiler (xkbcomp) reports:
> Error:            Cannot open "/var/lib/xkb/server-680.xkm" to write keyboard description
>                   Exiting
The XKEYBOARD keymap compiler (xkbcomp) reports:
> Error:            Cannot open "/var/lib/xkb/server-680.xkm" to write keyboard description
>                   Exiting
XKB: Failed to compile keymap
Keyboard initialization failed. This could be a missing or incorrect setup of xkeyboard-config.
(EE) 
Fatal server error:
(EE) Failed to activate virtual core keyboard: 2(EE) 
[1]  + Exit 1                        Xvfb :680 -screen 0 1024x768x24

If I execute

export CONT=/global/software/afni/containers/afni-23.1.10.sif

apptainer exec $CONT afni -ver

# apptainer exec $CONT ls /opt/afni/install

apptainer exec $CONT afni_system_check.py -check_all

I get

Precompiled binary linux_ubuntu_16_64_glw_local_shared: Jun 30 2023 (Version AFNI_23.1.10 'Publius Helvius Pertinax')
-------------------------------- general ---------------------------------
architecture:         64bit ELF
cpu type:             x86_64
system:               Linux
release:              4.18.0-425.10.1.el8_7.x86_64
version:              #1 SMP Thu Jan 12 16:32:13 UTC 2023
distribution:         Ubuntu 18.04 bionic
number of CPUs:       1
apparent login shell: SHELL_NOT_DETECTED
shell RC file:        NONE (does not exist)

--------------------- AFNI and related program tests ---------------------
which afni           :
                     : AFNI_23.1.10 'Publius Helvius Pertinax'
AFNI_version.txt     : AFNI_23.1.10, linux_ubuntu_16_64_glw_local_shared, Jun 30 2023, local
which python         :
which R              :

instances of various programs found in PATH:
    afni    : 1   (/opt/afni/install/afni)
    R       : 1   (/usr/bin/R)
    python  : 1   (/usr/bin/python3.6)
    python2 : 0 
    python3 : 1   (/usr/bin/python3.6)

** have python3 but not python2

testing ability to start various programs...
    afni                 : success
    suma                 : success
    3dSkullStrip         : success
    3dAllineate          : success
    3dRSFC               : success
    SurfMesh             : success
    3dClustSim           : success
    uber_subject.py      : success
    3dMVM                : success

------------------------ dependent program tests -------------------------
checking for dependent programs...

which tcsh           :
which Xvfb           :

checking for R packages...
    rPkgsInstall -pkgs ALL -check : FAILURE
        
        oo Warning: 
           These packages are not installed on the computer: afex!
        These packages are not installed on the computer: phia!
        These packages are not installed on the computer: snow!
        These packages are not installed on the computer: lmerTest!
        These packages are not installed on the computer: paran!
        These packages are not installed on the computer: psych!
        These packages are not installed on the computer: brms!
        These packages are not installed on the computer: corrplot!
        These packages are not installed on the computer: metafor!
        

R RHOME : /usr/lib/R

------------------------------ python libs -------------------------------

++ module loaded: matplotlib.pyplot
   module file : /opt/user_pip_packages/lib/python3.6/site-packages/matplotlib/pyplot.py
   matplotlib version : 3.3.4

-------------------------------- env vars --------------------------------
PATH = /opt/afni/src/../install:/opt/cmake/cmake-3.14.7-Linux-x86_64/bin:/opt/user_pip_packages/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

PYTHONPATH = 
R_LIBS = 
LD_LIBRARY_PATH = /.singularity.d/libs

DYLD_LIBRARY_PATH = 
DYLD_FALLBACK_LIBRARY_PATH = 

----------------------------- eval dot files -----------------------------

----------- AFNI $HOME files -----------

    .afnirc                   : missing
    .sumarc                   : missing
    .afni/help/all_progs.COMP : missing

--------- shell startup files ----------

   ** -shell_list: invalid shell SHELL_NOT_DETECTED
   ** error: failed to process options...
------------------------------ data checks -------------------------------
data dir : missing AFNI_data6
data dir : missing AFNI_demos
data dir : missing suma_demo
data dir : missing afni_handouts
atlas    : did not find TT_N27+tlrc

------------------------------ OS specific -------------------------------

have Ubuntu system: Ubuntu 18.04 bionic

=========================  summary, please fix:  =========================
*  just be aware: login shell 'SHELL_NOT_DETECTED', but our code examples use 'tcsh'
*  missing program: afni
*  missing program: python
*  missing program: R
*  failure under initial "AFNI and related program tests"
*  missing program: tcsh
*  missing program: Xvfb
*  missing R packages (see rPkgsInstall)
*  please run: cp /opt/afni/install/AFNI.afnirc ~/.afnirc
*  please run: "suma -update_env" for .sumarc
*  please run: apsearch -update_all_afni_help
*  failure running init_user_dotfiles.py -test
*  insufficient data for AFNI bootcamp
   (see "Prepare for Bootcamp" on install pages)
*  possibly missing atlases

While I think not running the X11 visualization is OK, I'm a bit surprised about the errors in the afni_check section?

Also, is there a workaround for running the visualization locally and the rest of the computationally heavy stuff on the cluster?

Thank you so much in advance!!!

2

Hello,

I am not sure, but it seems unlikely that the /tmp/.X11-unix directory would be visible from the container, which poses a problem for creating the APQC HTML reports that way.

  • rick
3

Hi-

I'm not sure if this helps totally, but those error messages resemble these when using Xvfb:
https://afni.nimh.nih.gov/pub/dist/doc/htmldoc/tutorials/auto_image/auto_%40chauffeur_afni.html#troubleshooting
You could try setting the Mac permissions for that temporary directory before starting (it should only need to be done once per system).

Both @SSwarper and afni_proc.py use Xvfb-running programs (like @chauffeur_afni) to do useful QC snapshotting. I'm not sure why SUMA would have the same issue.

--pt

4

FWIW, /tmp is visible from singularity containers at my site. I don't remember if that was the default, but I don't imaging I would have added it.
@maz.
to see if it is, you could do:

apptainer shell $CONT
ls /tmp

If the container can see your /tmp, you may need your administrator to make the .X11-unix folder

5

Hi all,

Thanks for your replies and help! The tmp does create a list but it doesn't include X11/Xfvb. My workaround is to do processing on the cluster and then build the QC reports locally which seems to work well enough :slight_smile:

Thanks!