we run most of our processing pipeline through MATLAB for several reasons, but a recent update to MATLAB has introduced an issue with the afni python scrips (e.g., 1d_tool.py).
for example,
unix(['1d_tool.py -help')
returns
env: python: No such file or directory
ans =
127
but, when run in zsh, the expected help text is returned.
I know this isn’t an afni specific issue, but I’m curious – has anyone come across this issue?
Caveat: I don’t use Matlab, so don’t have a deep answer here—maybe not even a helpful one, it might be best classified as “thinking out loud”.
I just tested running Matlab on our HPC cluster here, and running:
unix(['1d_tool.py -help'])
It worked successfully for me, so the issue doesn’t seem to be inherent to the program file.
My understanding is that the return code of 127 means that the system cannot find that command. You noted that in a zsh terminal, you can run “1d_tool.py” and see the program help. But I wonder if your Matlab is using a different shell, one that doesn’t have the AFNI binaries directory added to its path, perhaps (or, from that specific message, perhaps not even python)?
What is the output of each of the following for you:
A) to see what shell Matlab is using
unix(['echo $0'])
B) to see which python is there/available
unix(['which python'])
C) to see what version of python that is
unix(['python -V'])
D) the shebang of the Python programs in AFNI uses ‘/usr/bin/env …’, which in some system configurations can be different that the standard ‘python’ (because people set up aliases or other things):
The sh shell is used by default in Matlab. I tend to use the system() command instead of unix(), but I don’t think that will be critical here. Take a look at this previous post to see if the Matlab specific environment variables help.
Other AFNI scripts work. I run the following code to make sure AFNI and MATLAB are working together:
setenv('DYLD_FALLBACK_LIBRARY_PATH',[ '/Users/conradbn/abin:' getenv('DYLD_FALLBACK_LIBRARY_PATH') ])
% is the path to AFNI already set?
[afninotfound, unixoutput] = unix('3dcopy');
% if not, then we will look for it...
if afninotfound
% is AFNI in /abin?
[afninotfound, unixoutput] = unix('/abin/3dcopy');
if ~afninotfound % found it!
setenv('PATH', [getenv('PATH') ':/abin']);
disp(sprintf('Adding /abin to Matlab''s path (for AFNI).'));
end
end
if afninotfound
% is AFNI in ~/abin?
[afninotfound, unixoutput] = unix('~/abin/3dcopy');
if ~afninotfound % found it!
current_directory = pwd;
eval('cd ~');
afni_directory = [pwd '/abin'];
setenv('PATH', [getenv('PATH') ':' afni_directory]);
disp(sprintf(['Adding ' afni_directory ' to Matlab''s path (for AFNI).']));
eval(['cd ' current_directory]);
clear current_directory afni_directory;
end
end
if afninotfound
% is AFNI in ~/Documents/abin?
[afninotfound, unixoutput] = unix('~/Documents/abin/3dcopy');
if ~afninotfound % found it!
current_directory = pwd;
eval('cd ~');
afni_directory = [pwd '/Documents/abin'];
setenv('PATH', [getenv('PATH') ':' afni_directory]);
disp(sprintf(['Adding ' afni_directory ' to Matlab''s path (for AFNI).']));
eval(['cd ' current_directory]);
clear current_directory afni_directory;
end
end
if afninotfound
% if AFNI is not in /abin nor ~/abin, then the user will have to add
% the correct path to AFNI before this function will work.
disp(sprintf('*****'));
disp(sprintf('The required AFNI functions cannot be found.'));
disp(sprintf('AFNI needs to be added to Matlab''s path.'));
disp(sprintf('You can do this with the following command:'));
disp(sprintf(' '));
disp(sprintf(' setenv(''PATH'', [getenv(''PATH'') '':/abin'']);'));
disp(sprintf(' '));
disp(sprintf('where (as an example) ''/abin'' is the AFNI directory.'));
disp(sprintf('Exiting...'));
disp(sprintf('*****'));
return;
end
clear afninotfound unixoutput;
To answer your questions:
The unix env is in zsh.
>> unix(['echo $0'])
/bin/zsh
ans =
0
However, python is not recognized. This is interested because other versions of Matlab have worked.
>> unix(['which python'])
python not found
ans =
1
>> unix(['python -V'])
zsh:1: command not found: python
ans =
127
>> unix(['/usr/bin/env python -V'])
env: python: No such file or directory
ans =
127
I’ve run these on a different computer with MATLAB 2019a and I get the expected returns that are provided from terminal.
I suspect I need to add a few lines of code to my start up script above to map python to the unix/system env in MATLAB?
-------------------------------- general ---------------------------------
architecture: 64bit
system: Darwin
release: 21.4.0
version: Darwin Kernel Version 21.4.0: Mon Feb 21 20:34:37 PST 2022; root:xnu-8020.101.4~2/RELEASE_X86_64
distribution: 10.16
number of CPUs: 6
apparent login shell: zsh
shell RC file: .zshrc (exists)
--------------------- AFNI and related program tests ---------------------
which afni : /Users/andrewlynn/abin/afni
afni version : Precompiled binary macos_10.12_local: Dec 3 2021
: AFNI_21.3.10 'Trajan'
AFNI_version.txt : AFNI_21.3.10, macos_10.12_local, Dec 03 2021
which python : /Users/andrewlynn/miniconda3/bin/python
python version : 3.9.7
which R : /usr/local/bin/R
R version : R version 3.6.3 (2020-02-29) -- "Holding the Windsock"
which tcsh : /bin/tcsh
instances of various programs found in PATH:
afni : 1 (/Users/andrewlynn/abin/afni)
R : 1 (/Library/Frameworks/R.framework/Versions/3.6/Resources/bin/R)
python : 1 (/Users/andrewlynn/miniconda3/bin/python3.9)
python2 : 0
python3 : 3
/Users/andrewlynn/miniconda3/bin/python3.9
/Library/Frameworks/Python.framework/Versions/3.10/bin/python3.10
/usr/bin/python3
** have python3 but not python2
testing ability to start various programs...
afni : success
suma : success
3dSkullStrip : success
uber_subject.py : success
3dAllineate : success
3dRSFC : success
SurfMesh : success
3dClustSim : success
3dMVM : success
checking for R packages...
rPkgsInstall -pkgs ALL -check : success
R RHOME : /Library/Frameworks/R.framework/Resources
checking for $HOME files...
.afnirc : found
.sumarc : found
.afni/help/all_progs.COMP : found
------------------------------ python libs -------------------------------
** failed to load module PyQt4
-- PyQt4 is no longer needed for an AFNI bootcamp
** failed to load module matplotlib.pyplot
-- matplotlib.pyplot is not required, but is desirable
-- python binaries under /usr/local/bin:
/usr/local/bin/python3 (sym link to /Library/Frameworks/Python.framework/Versions/3.10/bin/python3.10)
/usr/local/bin/python3.9 (sym link to /usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/bin/python3.9)
-------------------------------- env vars --------------------------------
PATH = /Users/andrewlynn/miniconda3/bin:/Users/andrewlynn/miniconda3/condabin:/Library/Frameworks/Python.framework/Versions/3.10/bin:/Applications/freesurfer/7.1.1/bin:/Applications/freesurfer/7.1.1/fsfast/bin:/Applications/freesurfer/7.1.1/mni/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Library/Apple/usr/bin:/Users/andrewlynn/abin
PYTHONPATH =
R_LIBS =
LD_LIBRARY_PATH =
zsh compinit: insecure directories, run compaudit for list.
Ignore insecure directories and continue [y] or abort compinit [n]? y
DYLD_LIBRARY_PATH (sub-shell) = :/opt/X11/lib/flat_namespace
zsh compinit: insecure directories, run compaudit for list.
Ignore insecure directories and continue [y] or abort compinit [n]? DYLD_FALLBACK_LIBRARY_PATH (sub-shell) =
------------------------------ data checks -------------------------------
data dir : missing AFNI_data6
data dir : missing AFNI_demos
data dir : missing suma_demo
data dir : missing afni_handouts
atlas : found TT_N27+tlrc under /Users/andrewlynn/abin
------------------------------ OS specific -------------------------------
XQuartz version : 2.8.1
which brew : /usr/local/bin/brew
brew version : Homebrew 3.2.7
++ found PyQt4 under /usr/local/lib/python2.7/site-packages
** warning: have brew? PyQt4, but non-brew python /Users/andrewlynn/miniconda3/bin/python
-- consider installing gcc under homebrew
-- consider installing glib under homebrew
++ found 1 dylib files under '/opt/X11/lib/flat_namespace'
-- found 'libXt' dylib files:
/opt/X11/lib/flat_namespace/libXt.6.dylib
-- recent OS X, cheating to check DYLD_LIBRARY_PATH in cur shell 'zsh'...
zsh compinit: insecure directories, run compaudit for list.
Ignore insecure directories and continue [y] or abort compinit [n]? y
++ found evar DYLD_LIBRARY_PATH = :/opt/X11/lib/flat_namespace
-- recent OS X, cheating to check DYLD_LIBRARY_PATH in shell 'tcsh'...
** env var DYLD_LIBRARY_PATH not set to contain /opt/X11/lib/flat_namespace
========================= summary, please fix: =========================
* just be aware: login shell 'zsh', but our code examples use 'tcsh'
* insufficient data for AFNI bootcamp
(see "Prepare for Bootcamp" on install pages)
* please set DYLD_LIBRARY_PATH to /opt/X11/lib/flat_namespace in tcsh
… which should be totally unrelated to the present Python question.
Back to the Python stuff:
How do you start a conda environment in your normal shell usage? I think the problem lies with your matlab-called zsh doesn’t appear to have your conda environment started up.
Also, I would assume it would be easier to have the AFNI binaries added to your path from the shell setup (outside matlab), then to try to add/append path things in Matlab. But, based on your afni_system_check.py output, I guess that is the case? (Did you run that system_check from your matlab env, or from the terminal command line itself?)
The
National Institute of Mental Health (NIMH) is part of the National Institutes of
Health (NIH), a component of the U.S. Department of Health and Human
Services.