3dNwarpXYZ cannot read 1D files

Hello - I've had AFNI working on my Mac (M3, Sonoma) for a few months now, including a post-processing custom script that uses 3dNwarpXYZ to transform tlrc coordinates in 1D files into orig coordinates.

Now, we are getting an error with 3dNwarpXYZ saying that it cannot read the 1D files. I have confirmed that the file exists, and that I have full read/write/execute access to the 1D file.

Is there a specific python/R package that 3dNWarpXYZ uses that may have gone missing?

Output from afni_system_check.py -check_all :

-------------------------------- general ---------------------------------
architecture:     64bit
cpu type:       arm
system:        Darwin
release:       23.2.0
version:       Darwin Kernel Version 23.2.0: Wed Nov 15 21:54:55 PST 2023; root:xnu-10002.61.3~2/RELEASE_ARM64_T8122
distribution:     14.2.1
number of CPUs:    8
apparent login shell: zsh
shell RC file:    .zshrc (exists)
--------------------- AFNI and related program tests ---------------------
which afni      : /Users/hml/abin/afni
afni version     : Precompiled binary macos_13_ARM_clang: May 24 2024
           : AFNI_24.1.10 'Publius Septimius Geta'
AFNI_version.txt   : AFNI_24.1.10, macos_13_ARM_clang, May 24 2024, build
which python     : /Users/hml/.pyenv/versions/3.11.9/bin/python
python version    : 3.11.9
which R       : /usr/local/bin/R
R version      : R version 4.3.1 (aarch64-apple-darwin20)
instances of various programs found in PATH:
  afni  : 1  (/Users/hml/abin/afni)
  R    : 1  (/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/bin/R)
  python : 3
   /Users/hml/.pyenv/versions/3.11.9/bin/python3.11
   /Users/hml/.pyenv/shims/python
   /usr/bin/python3
  python2 : 0
  python3 : 3
   /Users/hml/.pyenv/versions/3.11.9/bin/python3.11
   /Users/hml/.pyenv/shims/python3
   /usr/bin/python3
** have python3 but not python2
testing ability to start various programs...
  afni         : success
  suma         : success
  3dSkullStrip     : success
  3dAllineate     : success
  3dRSFC        : success
  SurfMesh       : success
  3dClustSim      : success
  uber_subject.py   : success
  3dMVM        : success
------------------------ dependent program tests -------------------------
checking for dependent programs...
which tcsh      : /bin/tcsh
tcsh version     : 6.21.00
which Xvfb      : /opt/X11/bin/Xvfb
checking for R packages...
  rPkgsInstall -pkgs ALL -check : success
R RHOME : /Library/Frameworks/R.framework/Resources
------------------------------ python libs -------------------------------
++ module loaded: matplotlib.pyplot
  module file : /Users/hml/.pyenv/versions/3.11.9/lib/python3.11/site-packages/matplotlib/pyplot.py
  matplotlib version : 3.9.0
** failed to load module flask
-- flask is not required, but is desirable
** failed to load module flask_cors
-- flask_cors is not required, but is desirable
-- python binaries under /usr/local/bin:
  /usr/local/bin/python (sym link to /usr/bin/python3)
-------------------------------- env vars --------------------------------
PATH            = /Users/hml/.pyenv/versions/3.11.9/bin:/opt/homebrew/Cellar/pyenv/2.4.1/libexec:/opt/homebrew/Cellar/pyenv/2.4.1/plugins/python-build/bin:/Users/hml/.pyenv/shims:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/opt/X11/bin:/usr/local/munki:/Users/hml/Applications/SimNIBS-4.0/bin:/opt/homebrew/opt/python/libexec/bin:/Users/hml/abin
PYTHONPATH         =
R_LIBS           = /Users/hml/sw/R-4.3.1
LD_LIBRARY_PATH      =
DYLD_LIBRARY_PATH (sub-shell) =
DYLD_FALLBACK_LIBRARY_PATH (sub-shell) =
CONDA_SHLVL        =
CONDA_DEFAULT_ENV     =
----------------------------- eval dot files -----------------------------
----------- AFNI $HOME files -----------
  .afnirc          : found
  .sumarc          : found
  .afni/help/all_progs.COMP : found
--------- shell startup files ----------
  -- good: .tcshrc seems to contain 'source .cshrc'
  -- considered operations: path, apsearch
  -- note: followers should not need edits, so edit flags should be 0
   (have 1 follower(s), which can be ignored)
  no modifications needed across 3 dot files
------------------------------ data checks -------------------------------
data dir : missing AFNI_data6
data dir : missing AFNI_demos
data dir : missing suma_demo
data dir : missing afni_handouts
atlas  : found TT_N27+tlrc under /Users/hml/abin
------------------------------ OS specific -------------------------------
XQuartz version   : 2.8.5
which brew      : /opt/homebrew/bin/brew
brew version     : Homebrew 4.3.1
-- consider installing gcc under homebrew
-- consider installing glib under homebrew
========================= summary, please fix: =========================
* just be aware: login shell 'zsh', but our code examples use 'tcsh'
* insufficient data for AFNI bootcamp
  (see "Prepare for Bootcamp" on install pages)

Hmm, weird.

Can you please copy+paste:

  • your 3dNwarpXYZ command,
  • the full output terminal text,
  • and the 1D file content
    ?

That will help me try to replicate it and check.

thanks,
pt

echo "37 57 46" > s01.leftPAR.tlrc.1D

3dNwarpXYZ -nwarp 'anat.un.aff.qw_WARP.nii anat.un.aff.Xat.1D' s01.leftPAR.tlrc.1D > s01.leftPAR.orig.txt

terminal error:
++ 3dNwarpXYZ: AFNI version=AFNI_24.1.10 (May 24 2024) [64-bit]
++ Authored by: Zhark the Coordinated
** FATAL ERROR: Can't read 1D file 's01.leftPAR.tlrc.1D'
** Program compile date = May 24 2024

When I cat s01.leftPAR.tlrc.1D, it correctly shows the coordinates I fed it from the echo command.

The anat files are from the preproc stesp (auto tlrc and align to TT_N27)

Hmmm, is there a bad character somehow in that text file or something else? When I run the following similar command (same syntax, different filenames from using Bootcamp data) on my computer:

echo "37 57 46" > s01.leftPAR.tlrc.1D

3dNwarpXYZ \
   -nwarp 'anatQQ.FT_WARP.nii anatQQ.FT.aff12.1D' \
   s01.leftPAR.tlrc.1D \
   > s01.leftPAR.orig.txt

... the output is non-error and looks fine:

++ 3dNwarpXYZ: AFNI version=AFNI_24.1.19 (Jun 18 2024) [64-bit]
++ Authored by: Zhark the Coordinated

What is the output of ls -l on that first *.1D file for you? For me, it is:

ls -l s01.leftPAR.tlrc.1D
-rw-r--r--  1 taylorpa3  staff  12 Jun 27 10:09 s01.leftPAR.tlrc.1D

... so the permissions and all look fine.

I also tried running file_tool to check for bad characters, but there were none in my case---please check yours:

$ file_tool -show_bad_all -infiles s01.leftPAR.tlrc.1D 
s01.leftPAR.tlrc.1D has 0 bad characters

s01.leftPAR.tlrc.1D file type: UNIX

Fiinally, I'm not sure about the *Xat.1D file extension; I used a *.aff12.1D file.

--pt

It's very odd.

It works fine for me on other computers/different AFNI builds, which makes me think something happened to a dependency (I suspect a python package) with the way 3dNwarpXYZ reads the 1D files. Other AFNI commands using 1D files work fine (1D_tool.py, 3dTcoor1D).

Output from

ls -l s01.leftPAR.tlrc.1D
rw-r--r--  1 msh23m  staff  10 Jun 24 08:013 s01.leftPAR.tlrc.1D

file_tool also came back with 0 bad characters

It is odd. We tried this on a macOS with ARM, and it worked OK. (I had initially verified on my Ubuntu and macOS Intel machines and it worked on both; now we now it should work specifically on the OS you are using, too.)

Perhaps try updating your AFNI version, and see if that resolves it? Sorry that I don't have a deep explanation for why it doesn't appear to work in that limited case.

--pt

Sharing the solution here, in case others encounter it:

On one of my Macs, typing this on the terminal:
echo "37 57 46" > s01.leftPAR.tlrc.1D
resulted in a file without the quotes (i.e., just 37 57 46) and 3dNwarpXYZ could read that file.

On another Mac, the same echo command would save the file with the quote marks included, and 3dNwarpXYZ would fail to read that file with the quotes.

There are apparently different default echo versions/packages.

Wow, that is stunning, and thanks for sharing that!

Just to check, what happens on that Mac if you do:

\echo "37 57 46" > TEST.1D

? Does putting that slash there mean that the output TEST.1D does not contain quotes? I am wondering if the "echo" command might have been aliased on that computer---putting the slash before the command would use an un-aliased version.

thanks,
pt