weird error from auto_warp.py

AFNI version info (afni -ver):
Precompiled binary Linux_xorg7_64: Aug 16 2024 (Version AFNI_24.2.02 'Macrinus')

auto_warp.py -base MNI_caez_N27+tlrc -input ./MNI152_T1_2009c+tlrc

I recently ran into an error message from auto_warp.py saying "Error: Anatomical dset pre.anat.un_XYZ_OYmeqfZ3JqF7n9LNC7gDAA+tlrc.HEAD not found." However, looking into the folder of "awpy", this file does exist in there. To demonstrate this issue, I was simply warping the "MNI152_T1_2009c+tlrc" to the template of "MNI_caez_N27+tlrc"; and pasting the script output below:

[lzh@master2 test_anat]$ auto_warp.py -base MNI_caez_N27+tlrc -input ./MNI152_T1_2009c+tlrc
/usr/local/AFNI/linux_xorg7_64/afnipy/db_mod.py:6822: SyntaxWarning: invalid escape sequence '\_'
  "%s       -prefix %s\_REML%s\n"                 \
#++ auto_warp.py version: 0.06
# Output directory /data/home3/lzh/temp5/test_anat/awpy/
#Script is running:
  mkdir /data/home3/lzh/temp5/test_anat/awpy/
cd /data/home3/lzh/temp5/test_anat/awpy/
#Script is running (command trimmed):
  3dcopy /data/home3/lzh/temp5/test_anat/MNI152_T1_2009c+tlrc ./anat.nii
++ 3dcopy: AFNI version=AFNI_24.2.02 (Aug 16 2024) [64-bit]
#Script is running (command trimmed):
  3dUnifize -GM -input ./anat.nii -prefix ./anat.un.nii
++ 3dUnifize: AFNI version=AFNI_24.2.02 (Aug 16 2024) [64-bit]
 + Pre-processing: AD
 + Voxel loop: 0123456789.0123456789.0123456789.0123456789.0123456789.!UW[s1]Gm
++ Output dataset ./anat.un.nii
++ ===== Elapsed = 49.3 sec
#Script is running (command trimmed):
  3dcopy /usr/local/AFNI/linux_xorg7_64/MNI_caez_N27+tlrc ./base.nii
++ 3dcopy: AFNI version=AFNI_24.2.02 (Aug 16 2024) [64-bit]
#++ Aligning /data/home3/lzh/temp5/test_anat/awpy/base.nii data to /data/home3/lzh/temp5/test_anat/awpy/anat.un.nii data
#Script is running (command trimmed):
  @auto_tlrc -base ./base.nii -input ./anat.un.nii -suffix .aff -no_ss -no_pre -init_xform CENTER 

Notice:
Input dataset is in +tlrc already, @auto_tlrc might fail.
If the space of ./anat.un.nii is incorrectly set (check with next command)
   3dinfo -prefix -av_space ./anat.un.nii 
you can reset it with something like:
   3drefit -space ORIG ./anat.un.nii 
then remove all temp files, and try again.

Copying NIFTI volume to ./anat.un_XYZ_OYmeqfZ3JqF7n9LNC7gDAA
++ 3dcopy: AFNI version=AFNI_24.2.02 (Aug 16 2024) [64-bit]

Performing center alignment with @Align_Centers

++ 3dcopy: AFNI version=AFNI_24.2.02 (Aug 16 2024) [64-bit]
++ 3drefit: AFNI version=AFNI_24.2.02 (Aug 16 2024) [64-bit]
++ Authored by: RW Cox
++ Processing AFNI dataset ./anat.un_XYZ_OYmeqfZ3JqF7n9LNC7gDAA_shft+tlrc
 + deoblique
++ 3drefit processed 1 datasets
++ 3drename: AFNI version=AFNI_24.2.02 (Aug 16 2024) [64-bit]
** THD_rename_dataset_files: old header anat.un_XYZ_OYmeqfZ3JqF7n9LNC7gDAA_shft+orig.HEAD doesn't exist!
** THD_rename_dataset_files: old header anat.un_XYZ_OYmeqfZ3JqF7n9LNC7gDAA_shft+acpc.HEAD doesn't exist!
++ THD_rename_dataset_files: rename anat.un_XYZ_OYmeqfZ3JqF7n9LNC7gDAA_shft+tlrc.HEAD -> __ats_tmp__anat.un_XYZ_OYmeqfZ3JqF7n9LNC7gDAA_shft+tlrc.HEAD
++ THD_rename_dataset_files: rename anat.un_XYZ_OYmeqfZ3JqF7n9LNC7gDAA_shft+tlrc.BRIK -> __ats_tmp__anat.un_XYZ_OYmeqfZ3JqF7n9LNC7gDAA_shft+tlrc.BRIK
++ 3dWarp: AFNI version=AFNI_24.2.02 (Aug 16 2024) [64-bit]
++ Authored by: RW Cox
++ anat already stripped, re-using
Error: Anatomical dset pre.anat.un_XYZ_OYmeqfZ3JqF7n9LNC7gDAA+tlrc.HEAD not found.
#**ERROR Failed in affine step
** ERROR - script failed

But if I list the content of "awpy", the claimed missing file does exist:

[lzh@master2 test_anat]$ ls -l awpy/
total 145924
-rw-r--r--. 1 lzh lzh  8533509 Jan 19 10:57 anat.nii
-rw-r--r--. 1 lzh lzh 34123716 Jan 19 10:58 anat.un.nii
-rw-r--r--. 1 lzh lzh      244 Jan 19 10:58 anat.un_XYZ_OYmeqfZ3JqF7n9LNC7gDAA_shft.1D
-rw-r--r--. 1 lzh lzh 34120084 Jan 19 10:58 anat.un_XYZ_OYmeqfZ3JqF7n9LNC7gDAA+tlrc.BRIK
-rw-r--r--. 1 lzh lzh     3036 Jan 19 10:58 anat.un_XYZ_OYmeqfZ3JqF7n9LNC7gDAA+tlrc.HEAD
-rw-r--r--. 1 lzh lzh 34120084 Jan 19 10:58 __ats_tmp__anat.un_XYZ_OYmeqfZ3JqF7n9LNC7gDAA_shft+tlrc.BRIK
-rw-r--r--. 1 lzh lzh     3404 Jan 19 10:58 __ats_tmp__anat.un_XYZ_OYmeqfZ3JqF7n9LNC7gDAA_shft+tlrc.HEAD
-rw-r--r--. 1 lzh lzh  4375336 Jan 19 10:58 base.nii
-rw-r--r--. 1 lzh lzh 34120084 Jan 19 10:58 pre.anat.un_XYZ_OYmeqfZ3JqF7n9LNC7gDAA+tlrc.BRIK
-rw-r--r--. 1 lzh lzh     3858 Jan 19 10:58 pre.anat.un_XYZ_OYmeqfZ3JqF7n9LNC7gDAA+tlrc.HEAD

So what could go wrong here?

Hi, Zhihao-

I see this error/warning message in line 3 out the output text there:

  "%s       -prefix %s\_REML%s\n"                 \

I checked in the repository about that, and that was actually fixed in this PR on Sept 30, 2024.

I am running your command now to see if there are any issues still (my code version is modern enough to include that fix), and so far things are OK (into lev=4 of the warping).

I will let you know if any other error pops up, but I suspect if you update your local AFNI version, then this issue should be resolved.

--pt

ps: there is this warning in the text terminal, as well:

Input dataset is in +tlrc already, @auto_tlrc might fail.

... and i had wondered if this would present a problem, but we will see...

Hi again, Zhihao-

And as a followup, auto_warp.py appears to have completed successfully for me.

So, indeed, the issue appears to be fixed in more recent AFNI.

--pt

Thank you Paul, for your test and guidance! My system administrator has updated AFNI for me, but the same issue still persists. Could you please take another look at the following outputs? Thank you so much for your help!

Check AFNI version:

[lzh@master2 test_anat]$ which afni
/usr/local/afni_OpenMP/linux_openmp_64/afni
[lzh@master2 test_anat]$ afni -ver
Precompiled binary linux_openmp_64: Jan 14 2025 (Version AFNI_25.0.00 'Severus Alexander')

Replicate the issue:

[lzh@master2 test_anat]$ auto_warp.py -base MNI_caez_N27+tlrc -input ./MNI152_T1_2009c+tlrc
#++ auto_warp.py version: 0.06
# Output directory /data/home3/lzh/temp5/test_anat/awpy/
#Script is running:
  mkdir /data/home3/lzh/temp5/test_anat/awpy/
cd /data/home3/lzh/temp5/test_anat/awpy/
#Script is running (command trimmed):
  3dcopy /data/home3/lzh/temp5/test_anat/MNI152_T1_2009c+tlrc ./anat.nii
++ 3dcopy: AFNI version=AFNI_25.0.00 (Jan 14 2025) [64-bit]
#Script is running (command trimmed):
  3dUnifize -GM -input ./anat.nii -prefix ./anat.un.nii
++ 3dUnifize: AFNI version=AFNI_25.0.00 (Jan 14 2025) [64-bit]
 + Pre-processing: ADV.....UW[s1]Gm
++ Output dataset ./anat.un.nii
++ ===== Elapsed = 10.5 sec
#Script is running (command trimmed):
  3dcopy /usr/local/afni_OpenMP/linux_openmp_64/MNI_caez_N27+tlrc ./base.nii
++ 3dcopy: AFNI version=AFNI_25.0.00 (Jan 14 2025) [64-bit]
#++ Aligning /data/home3/lzh/temp5/test_anat/awpy/base.nii data to /data/home3/lzh/temp5/test_anat/awpy/anat.un.nii data
#Script is running (command trimmed):
  @auto_tlrc -base ./base.nii -input ./anat.un.nii -suffix .aff -no_ss -no_pre -init_xform CENTER 

Notice:
Input dataset is in +tlrc already, @auto_tlrc might fail.
If the space of ./anat.un.nii is incorrectly set (check with next command)
   3dinfo -prefix -av_space ./anat.un.nii 
you can reset it with something like:
   3drefit -space ORIG ./anat.un.nii 
then remove all temp files, and try again.

Copying NIFTI volume to ./anat.un_XYZ_OYmeqfZ3JqF7n9LNC7gDAA
++ 3dcopy: AFNI version=AFNI_25.0.00 (Jan 14 2025) [64-bit]

Performing center alignment with @Align_Centers

++ 3dcopy: AFNI version=AFNI_25.0.00 (Jan 14 2025) [64-bit]
++ 3drefit: AFNI version=AFNI_25.0.00 (Jan 14 2025) [64-bit]
++ Authored by: RW Cox
++ Processing AFNI dataset ./anat.un_XYZ_OYmeqfZ3JqF7n9LNC7gDAA_shft+tlrc
 + deoblique
++ 3drefit processed 1 datasets
++ 3drename: AFNI version=AFNI_25.0.00 (Jan 14 2025) [64-bit]
** THD_rename_dataset_files: old header anat.un_XYZ_OYmeqfZ3JqF7n9LNC7gDAA_shft+orig.HEAD doesn't exist!
** THD_rename_dataset_files: old header anat.un_XYZ_OYmeqfZ3JqF7n9LNC7gDAA_shft+acpc.HEAD doesn't exist!
++ THD_rename_dataset_files: rename anat.un_XYZ_OYmeqfZ3JqF7n9LNC7gDAA_shft+tlrc.HEAD -> __ats_tmp__anat.un_XYZ_OYmeqfZ3JqF7n9LNC7gDAA_shft+tlrc.HEAD
++ THD_rename_dataset_files: rename anat.un_XYZ_OYmeqfZ3JqF7n9LNC7gDAA_shft+tlrc.BRIK -> __ats_tmp__anat.un_XYZ_OYmeqfZ3JqF7n9LNC7gDAA_shft+tlrc.BRIK
++ 3dWarp: AFNI version=AFNI_25.0.00 (Jan 14 2025) [64-bit]
++ Authored by: RW Cox
++ anat already stripped, re-using
Error: Anatomical dset pre.anat.un_XYZ_OYmeqfZ3JqF7n9LNC7gDAA+tlrc.HEAD not found.
#**ERROR Failed in affine step
** ERROR - script failed

Check the "awpy" folder:

[lzh@master2 test_anat]$ ls -l awpy/
total 145924
-rw-r--r--. 1 lzh lzh  8533509 Jan 25 10:52 anat.nii
-rw-r--r--. 1 lzh lzh 34123716 Jan 25 10:52 anat.un.nii
-rw-r--r--. 1 lzh lzh      244 Jan 25 10:52 anat.un_XYZ_OYmeqfZ3JqF7n9LNC7gDAA_shft.1D
-rw-r--r--. 1 lzh lzh 34120084 Jan 25 10:52 anat.un_XYZ_OYmeqfZ3JqF7n9LNC7gDAA+tlrc.BRIK
-rw-r--r--. 1 lzh lzh     3039 Jan 25 10:52 anat.un_XYZ_OYmeqfZ3JqF7n9LNC7gDAA+tlrc.HEAD
-rw-r--r--. 1 lzh lzh 34120084 Jan 25 10:52 __ats_tmp__anat.un_XYZ_OYmeqfZ3JqF7n9LNC7gDAA_shft+tlrc.BRIK
-rw-r--r--. 1 lzh lzh     3409 Jan 25 10:52 __ats_tmp__anat.un_XYZ_OYmeqfZ3JqF7n9LNC7gDAA_shft+tlrc.HEAD
-rw-r--r--. 1 lzh lzh  4375544 Jan 25 10:52 base.nii
-rw-r--r--. 1 lzh lzh 34120084 Jan 25 10:52 pre.anat.un_XYZ_OYmeqfZ3JqF7n9LNC7gDAA+tlrc.BRIK
-rw-r--r--. 1 lzh lzh     3862 Jan 25 10:52 pre.anat.un_XYZ_OYmeqfZ3JqF7n9LNC7gDAA+tlrc.HEAD