@chauffeur_afni giving xvfb error on macOS 10.13.6

jsaletin · July 19, 2018, 6:08pm

Hi AFNI folks!

We recently upgraded OS; reinstalled AFNI and XQuartz and all seems to be ok, except for trying to run @chauffeur_afni as part of some fat_proc_convert scripts.

@chauffeur_afni fails to open xvfb – mentioning no display.

I’ve replicated the error outside of fatcat by calling @chauffeur_afni directly.


++ My command:
   @chauffeur_afni -ulay /Users/jared/abin/TT_152_2009c+tlrc.HEAD -prefix pretty -montx 5 -monty 3 -do_clean
\n------------------ start of optionizing ------------------\n
+* Noting: you did NOT provide an overlay, and yet
           you did NOT turn off the overlay with '-olay_off'
       --> so I assume you just want to view the underlay.
++ Using blowup factor: 2
++ Using opacity:  6
++ Making temporary work directory to copy vis files: ./__tmp_chauf_7nIKCMxqfqR
++ Copying ulay to visualize between 2%-98% (volumetric):
   [79.000000,236.000000]
++ 3dcalc: AFNI version=AFNI_18.2.04 (Jul  6 2018) [64-bit]
++ Authored by: A cast of thousands
++ Output dataset ././__tmp_chauf_7nIKCMxqfqR/tmp_ulay.nii
++ No overlay viewing
++ Cryptic info: RAI -> R A I -> 1 2 3
++ Dimensions (xyzt): 161 191 151 1
++ Slice spacing ordered (x,y,z) is:  10 12 10
++ pfrac = .50000
++ Will have: SET_IJK 80 95 75
\n------------------- end of optionizing -------------------\n
usage: count <expected line count>
usage: count <expected line count>
 -- trying to start Xvfb :
[1] 39862
Bad display name:
use: X [:<display>] [option]

AFNI itself loads appropriately and opens X just fine.

Has anyone run into this?

My AFNI_SYSTEM_CHECK is printed below…

Thanks!
Jared


-------------------------------- general ---------------------------------
architecture:         64bit 
system:               Darwin
release:              17.7.0
version:              Darwin Kernel Version 17.7.0: Thu Jun 21 22:53:14 PDT 2018; root:xnu-4570.71.2~1/RELEASE_X86_64
distribution:         10.13.6 x86_64
number of CPUs:       8
apparent login shell: zsh
shell RC file:        NONE (does not exist)

--------------------- AFNI and related program tests ---------------------
which afni           : /Users/jared/abin/afni
afni version         : Precompiled binary macosx_10.7_local: Jul  6 2018 
                     : AFNI_18.2.04
AFNI_version.txt     : AFNI_18.2.04, macosx_10.7_local, Jul 06 2018
which python         : /usr/local/bin/python
python version       : 2.7.15
which R              : /usr/local/bin/R
R version            : R version 3.5.1 (2018-07-02) -- "Feather Spray"
which tcsh           : /bin/tcsh

instances of various programs found in PATH:
    afni    : 1   (/Users/jared/abin/afni)
    R       : 1   (/usr/local/Cellar/r/3.5.1/bin/R)
    python  : 2 
      /usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/bin/python2.7
      /usr/bin/python
    python2 : 1   (/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/bin/python2.7)
    python3 : 1   (/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/bin/python3.7)


testing ability to start various programs...
    afni                 : success
    suma                 : success
    3dSkullStrip         : success
    uber_subject.py      : success
    3dAllineate          : success
    3dRSFC               : success
    SurfMesh             : success
    3dClustSim           : success

checking for R packages...
    rPkgsInstall -pkgs ALL -check : success

checking for $HOME files...
    .afnirc                   : found
    .sumarc                   : found
    .afni/help/all_progs.COMP : found

------------------------------ python libs -------------------------------
++ module 'PyQt4' found at /usr/local/lib/python2.7/site-packages/PyQt4
++ module loaded: PyQt4

-------------------------------- env vars --------------------------------
PATH = /usr/local/opt/llvm/bin:/Applications/TORTOISE_V3.1.2/DRTAMASV312/bin:/Applications/TORTOISE_V3.1.2/DRBUDDIV312/bin:/Applications/TORTOISE_V3.1.2/DIFFCALC/DIFFCALCV312:/Applications/TORTOISE_V3.1.2/DIFFPREPV312/bin/bin:/Applications/freesurfer/bin:/Applications/freesurfer/fsfast/bin:/Applications/freesurfer/tktools:/usr/local/fsl/bin:/Applications/freesurfer/mni/bin:/usr/local/fsl/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Users/jared/abin

PYTHONPATH = 
R_LIBS = 
LD_LIBRARY_PATH = 
DYLD_LIBRARY_PATH (sub-shell) = /Applications/freesurfer/lib/gcc/lib:/opt/X11/lib/flat_namespace
DYLD_FALLBACK_LIBRARY_PATH (sub-shell) = 

------------------------------ data checks -------------------------------
data dir : missing AFNI_data6
data dir : missing AFNI_demos
data dir : missing suma_demo
data dir : missing afni_handouts
atlas    : found TT_N27+tlrc  under /Users/jared/abin

------------------------------ OS specific -------------------------------
which brew           : /usr/local/bin/brew
brew version         : Homebrew 1.7.0

++ found valid link /usr/local/lib/libgomp.1.dylib
   to /usr/local/Cellar/gcc/8.1.0/lib/gcc/8/libgomp.1.dylib
++ found valid link /usr/local/lib/libglib-2.0.dylib
   to ../Cellar/glib/2.56.1/lib/libglib-2.0.dylib
++ found 1 dylib files under '/opt/X11/lib/flat_namespace'
   -- found 'libXt' dylib files:
      /opt/X11/lib/flat_namespace/libXt.6.dylib
-- recent OS X, cheating to check DYLD_LIBRARY_PATH in cur shell 'zsh'...
++ found evar DYLD_LIBRARY_PATH = /Applications/freesurfer/lib/gcc/lib:/opt/X11/lib/flat_namespace
-- recent OS X, cheating to check DYLD_LIBRARY_PATH in shell 'tcsh'...
++ found evar DYLD_LIBRARY_PATH = /opt/X11/lib/flat_namespace

=========================  summary, please fix:  =========================
*  login shell 'zsh', trusting user to translate from 'tcsh'
*  insufficient data for AFNI bootcamp

ptaylor · July 19, 2018, 8:29pm

I’m afraid I’ve never seen that before, and xvfb should run fine on that Mac OS version.

What is the output of:


echo $DISPLAY

?

–pt

jsaletin · July 19, 2018, 9:30pm

Yeah it’s perplexing. I even uninstalled and reinstalled XQuartz, restarted, etc…

I ran the echo command on a fresh terminal and got this as output. Things look in order?


/private/tmp/com.apple.launchd.pCtOPf4iYc/org.macosforge.xquartz:0

Thanks!
Jared

ptaylor · July 19, 2018, 9:53pm

Well, even though that looks odd to me (on my Ubuntu computer, I just get “:0”), I did a test on a colleague’s Mac computer-- “echo $DISPLAY” gave a similar-looking result, and she could still run @snapshot_volreg (which uses xvfb in the same way that @chauffeur* does).

Sooo… I don’t know.

I notice you are using “zsh” as your terminal shell instead of “tcsh” or “bash”, and that the ASC states that there doesn’t appear to be an RC file for that shell: “shell RC file: NONE (does not exist)”; though, your environment variables do appear set to the ASC. Was this the same setup you had previously?

–pt

jsaletin · July 20, 2018, 2:14pm

We do use ZSH, but it’s never given us trouble, as long as the path is correct.

I’ve pasted our OLD afni_system_check output below. Everything used to work fine (we use the fat_proc_* commands to do preprocessing of structural and dwi data).

Though the RC file is still NONE on the old one, so I’m not sure.

Save trying to clear everything off and reinstalling all key software from scratch (already reinstalled X)

~Jared


-------------------------------- general ---------------------------------
architecture:         64bit
system:               Darwin
release:              17.6.0
version:              Darwin Kernel Version 17.6.0: Tue May  8 15:22:16 PDT 2018; root:xnu-4570.61.1~1/RELEASE_X86_64
distribution:         10.13.5 x86_64
number of CPUs:       8
apparent login shell: zsh
shell RC file:        NONE (does not exist)

--------------------- AFNI and related program tests ---------------------
which afni           : /Users/jared/abin/afni
afni version         : Precompiled binary macosx_10.7_local: Nov  9 2017
                     : AFNI_17.3.03
AFNI_version.txt     : AFNI_17.3.03, macosx_10.7_local, Nov 09 2017
which python         : /usr/local/bin/python
python version       : 2.7.13
which R              : /usr/local/bin/R
R version            : R version 3.2.3 (2015-12-10) -- "Wooden Christmas-Tree"
which tcsh           : /bin/tcsh

instances of various programs found in PATH:
    afni    : 1   (/Users/jared/abin/afni)
    R       : 1   (/Library/Frameworks/R.framework/Versions/3.2/Resources/bin/R)
    python  : 3
      /opt/local/bin/python2.7
      /usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/bin/python2.7
      /usr/bin/python
    python2 : 1   (/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/bin/python2.7)
    python3 : 1   (/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/bin/python3.6)


testing ability to start various programs...
    afni                 : success
    suma                 : success
    3dSkullStrip         : success
    uber_subject.py      : success
    3dAllineate          : success
    3dRSFC               : success
    SurfMesh             : success
    3dClustSim           : success

checking for R packages...
    rPkgsInstall -pkgs ALL -check : FAILURE
           	This package has been verified on the computer: afex
        	This package has been verified on the computer: phia
        	This package has been verified on the computer: snow
        	This package has been verified on the computer: nlme
        	This package has been verified on the computer: lme4
        	This package has been verified on the computer: paran
        	This package has been verified on the computer: psych

        Warning message:
        In as.POSIXlt.POSIXct(x, tz) : unknown timezone 'default/America/New_York'

checking for $HOME files...
    .afnirc                   : found
    .sumarc                   : found
    .afni/help/all_progs.COMP : found

------------------------------ python libs -------------------------------
++ module 'PyQt4' found at /usr/local/lib/python2.7/site-packages/PyQt4
++ module loaded: PyQt4

-------------------------------- env vars --------------------------------
PATH = /Applications/MATLAB_R2017a.app/bin:/Applications/TORTOISE_V3.1.0/DRBUDDIV31/bin:/Applications/TORTOISE_V3.1.0/DIFFCALC/DIFFCALCV31:/Applications/TORTOISE_V3.1.0/DIFFPREPV31/bin/bin:/Applications/Postgres.app/Contents/Versions/9.5/bin/:/usr/local/mysql/bin/:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/Applications/freesurfer/bin:/Applications/freesurfer/fsfast/bin:/Applications/freesurfer/tktools:/usr/local/fsl/bin:/Applications/freesurfer/mni/bin:/usr/local/fsl/bin:/Users/jared/abin:/Users/jared/antsbin/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/git/bin:/usr/texbin

PYTHONPATH = /usr/local/lib/python2.7/site-packages:

R_LIBS =
LD_LIBRARY_PATH =
DYLD_LIBRARY_PATH (sub-shell) = /Applications/TORTOISE_V3.1.0/DRBUDDIV31/bin:/opt/X11/lib/flat_namespace
DYLD_FALLBACK_LIBRARY_PATH (sub-shell) = /Users/jared/abin

------------------------------ data checks -------------------------------
data dir : missing AFNI_data6
data dir : missing AFNI_demos
data dir : missing suma_demo
data dir : missing afni_handouts
atlas    : found TT_N27+tlrc  under /Users/jared/abin

------------------------------ OS specific -------------------------------
which brew           : /usr/local/bin/brew
brew version         : Homebrew 1.3.6-109-g0cec599
which port           : /opt/local/bin/port
port version         : Error: Current platform "darwin 17" does not match expected platform "darwin 16"

++ found valid link /usr/local/lib/libgomp.1.dylib
   to /usr/local/Cellar/gcc/7.2.0/lib/gcc/7/libgomp.1.dylib
++ found valid link /usr/local/lib/libglib-2.0.dylib
   to ../Cellar/glib/2.52.0/lib/libglib-2.0.dylib
++ found 1 dylib files under '/opt/X11/lib/flat_namespace'
   -- found 'libXt' dylib files:
      /opt/X11/lib/flat_namespace/libXt.6.dylib
-- recent OS X, cheating to check DYLD_LIBRARY_PATH in cur shell 'zsh'...
++ found evar DYLD_LIBRARY_PATH = /Applications/TORTOISE_V3.1.0/DRBUDDIV31/bin:/opt/X11/lib/flat_namespace
-- recent OS X, cheating to check DYLD_LIBRARY_PATH in shell 'tcsh'...
** env var DYLD_LIBRARY_PATH not set to contain /opt/X11/lib/flat_namespace

=========================  summary, please fix:  =========================
*  login shell 'zsh', trusting user to translate from 'tcsh'
*  missing R packages (see rPkgsInstall)
*  insufficient data for AFNI bootcamp
*  please set DYLD_LIBRARY_PATH to /opt/X11/lib/flat_namespace in tcsh

ptaylor · July 20, 2018, 4:48pm

Hi, Jared-

Yes, reinstalling all sounds drastic.

Some straw grasping:

if you type “ps”, you don’t have any Xvfb jobs that are still hanging around from previous runs, by any chance?
What is the output of


ls /tmp/.X*lock

on your computer?

To see where exactly this is failing, the following is a stripped-down version of @chauffeur*, namely the Xvfb setup trying to find an available display:


#!/bin/tcsh

set ranval = `count -dig 1 1 999999 R1`
echo "MSG_AA.  ranval = $ranval"

if( $?xdisplay == 0 )then
    set killX     = 1
    set ntry      = 1
    set Xnotfound = 1
    while( $Xnotfound )
        set xdisplay = `count -dig 1 3 999 R1`

        echo "MSG_BB.  xdisplay = $xdisplay"
        if( -e /tmp/.X${xdisplay}-lock ) then
            echo "MSG_CC.  Found tmpfile = /tmp/.X${xdisplay}-lock"
        endif

        if( -e /tmp/.X${xdisplay}-lock ) continue
        echo " -- trying to start Xvfb :${xdisplay}"
        Xvfb :${xdisplay} -screen 0 1024x768x24 &
        sleep 1
        jobs > zzerm.$ranval.txt
        grep -q Xvfb zzerm.$ranval.txt
        set Xnotfound = $status

        echo "MSG_DD.  status = $status"

        cat zzerm.$ranval.txt

        \rm -f zzerm.$ranval.txt
        if( $Xnotfound == 0 )break ;
        @ ntry++
        if( $ntry > 99 )then
            echo "** ERROR: can't start Xvfb -- exiting"
            exit 1
        endif
    end
endif

setenv DISPLAY :${xdisplay}

# A la bob:  stop Xvfb if we started it ourselves
if( $?killX ) kill %1

echo "\n\n\n\n\nDONE. Bye.\n\n\n\n"

If you save that to a text file called “do_teset_xvfb.tcsh” and run:


tcsh -ef do_test_xvfb.tcsh > asdfasdf.txt

what are the contents of asdfasdf.txt? For me, it is just a once through:


MSG_AA.  ranval = 977331
MSG_BB.  xdisplay = 357
 -- trying to start Xvfb :357
[1] 703
MSG_DD.  status = 0
[1]  + Running                       Xvfb :357 -screen 0 1024x768x24





DONE. Bye.

–pt

rickr · July 20, 2018, 5:32pm

Hi Jared,

This output states “please set DYLD_LIBRARY_PATH to /opt/X11/lib/flat_namespace in tcsh”.

Do you have that variable set in your .cshrc/.tcshrc file?

rick

ptaylor · July 20, 2018, 7:11pm

Well, in the old ASC output, it looks unset, but in the newer one (earlier in thread) it does appear to be set?


DYLD_LIBRARY_PATH (sub-shell) = /Applications/freesurfer/lib/gcc/lib:/opt/X11/lib/flat_namespace
DYLD_FALLBACK_LIBRARY_PATH (sub-shell) =

…


-- recent OS X, cheating to check DYLD_LIBRARY_PATH in cur shell 'zsh'...
++ found evar DYLD_LIBRARY_PATH = /Applications/freesurfer/lib/gcc/lib:/opt/X11/lib/flat_namespace
-- recent OS X, cheating to check DYLD_LIBRARY_PATH in shell 'tcsh'...
++ found evar DYLD_LIBRARY_PATH = /opt/X11/lib/flat_namespace

Is that looking fine?

–pt

rickr · July 20, 2018, 7:54pm

Yes, that looks good. Of course, Jared could temporarily add
it to @chauffeur_afni, just to be more certain.

rick

ptaylor · July 20, 2018, 8:22pm

That’s a good idea, indeed.

–pt

jsaletin · July 20, 2018, 9:44pm

Thanks Paul!

So I ran the ps command did see one xvfb job, on the same terminal I had tried to run chauffeur with:


41926 ttys003    0:00.09 xvfb

running the ls command yielded:


/tmp/.X0-lock /tmp/.X5-lock

I rm’d both lock files and tried running chauffeur again to no success; still failed.

Running your test code gave a strange error on the count command:


usage: count <expected line count>

Perhaps something else is wrong with my set up I’m not accounting for?

Thank you to both you and Rick for helping sleuth this mystery!

~Jared

jsaletin · July 20, 2018, 9:46pm

Thanks Rick,

Add the set DYLD path command directly to the top of the Chauffeur script, you mean?

~Jared

ptaylor · July 20, 2018, 10:34pm

Yes, that is what Rick was suggesting; you could do that in the do_test*tcsh program, too, for testing purpose at the moment.


setenv DYLD_LIBRARY_PATH /opt/X11/lib/flat_namespace

Re. the above “count” error-- that is a bit funny, because “count” is program distributed with AFNI-- you can see this by typing:


which count

and it should point you at your AFNI binary location-- from your ASC, I think that should return “/Users/jared/abin/afni”. Does it?

In your zsh terminal, what does the following give:


count -dig 1 1 999999 R1

what does this give (-> switching shells; you can type “exit” afterward to leave tcsh):


tcsh
count -dig 1 1 999999 R1

–pt

ptaylor · July 20, 2018, 10:41pm

I have replied to the other prong of this thread below with some more things to try about this:
https://afni.nimh.nih.gov/afni/community/board/read.php?1,158709,158725#msg-158725

… but to note here:

I’m not sure that you have to remove those /tmp/*lock files, but OK (I think).
I think you should stop the other xvfb process, just so it isn’t running in background (you can “kill” is, after running “ps” to see its PID).
the problem might be with ‘count’ and not xvfb, actually; see the other post from above-- ‘count’ is program distributed with AFNI (“Written by RWCox back in the ancient mists of forgotten time”), so this would surprise me, since you say your AFNI stuff is working, but let’s see…

–pt

jsaletin · July 23, 2018, 2:04pm

I think we found it!

I ran the which count command and returned:


/usr/local/opt/llvm/bin/count

So somehow count was clearly being confused and not running the afni version. I switched my path and things seem to be running ok, for now!

Thanks so much, in helping work through this!

~Jared

ptaylor · July 23, 2018, 2:24pm

Phhhhhhhhheeeeeeeeeeeeeeeew. Good to know.

Out of curiosity, what is that interloping “count” program? Whence does it come, and with what disruptive purpose, according to its help?

–pt

jsaletin · July 23, 2018, 5:56pm

LLVM is a compiler support toolbox, I’m not sure what It’s used for, or how it got in my path, frankly.

But I’m glad we found it!

Thanks again!

ptaylor · July 23, 2018, 6:03pm

I googled a bit, and I think it comes with xcode; I don’t know whether it is a particular option flag the puts it there, but we will try to keep an eye out for it…

Anyways, happy chauffeuring now.

–pt

dglen · July 24, 2018, 12:27pm

I think this may have come from a homebrew install of llvm based on this bit I found. count is one of our oldest programs. It’s also very simply named, but used in many scripts, so I hope this doesn’t become more of an issue.

brew install llvm
Once you’re done with that, Homebrew will tell you something like this:

To use the bundled libc++ please add the following LDFLAGS:
LDFLAGS=“-L/usr/local/opt/llvm/lib -Wl,-rpath,/usr/local/opt/llvm/lib”

This formula is keg-only, which means it was not symlinked into /usr/local,
because macOS already provides this software and installing another version in
parallel can cause all kinds of trouble.

If you need to have this software first in your PATH run:
echo ‘export PATH=“/usr/local/opt/llvm/bin:$PATH”’ >> ~/.bash_profile

For compilers to find this software you may need to set:
LDFLAGS: -L/usr/local/opt/llvm/lib
CPPFLAGS: -I/usr/local/opt/llvm/include

If you need Python to find bindings for this keg-only formula, run:
echo /usr/local/opt/llvm/lib/python2.7/site-packages >> /usr/local/lib/python2.7/site-packages/llvm.pth