Issues When Running afni_proc.py

AFNI version info (afni -ver): AFNI_23.2.11

Hi there! I'm receiving some issues when trying to run afni_proc.py. The first issue is, when I run afni_proc.py, it thinks that there is only 1 run to analyze in the anat and func images I provided, but there are actually two runs. Unfortunately, the scanner was never stopped in between runs. Second, I keep receiving the warning message that -cmass is turned off, however, I thought align_opts_aea turned cmass on? Lastly, I am receiving this error in my output, that stops afni_proc.py from fully running:

3dAllineate -base anatSS.ATN001v1_ON_ns+orig -input rm.epi.all1+tlrc -1Dmatrix_apply mat.r01.warp.aff12.1D -mast_dxyz 2 -final NN -quiet -prefix rm.epi.1.r01
++ 3dAllineate: AFNI version=AFNI_23.2.11 (Sep 26 2023) [64-bit]
++ Authored by: Zhark the Registrator
e[7m*+ WARNING:e[0m   If you are performing spatial transformations on an oblique dset,
  such as ./rm.epi.all1+tlrc.BRIK,
  or viewing/combining it with volumes of differing obliquity,
  you should consider running: 
     3dWarp -deoblique 
  on this and  other oblique datasets in the same session.
 See 3dWarp -help for details.
++ Oblique dataset:./rm.epi.all1+tlrc.BRIK is 19.635971 degrees from plumb.
++ NOTE: base and source coordinate systems have different handedness
 +       Orientations: base=Right handed (LPI); source=Left handed (RPI)
 +     - It is nothing to worry about: 3dAllineate aligns based on coordinates.
 +     - But it is always important to check the alignment visually to be sure.
e[7m*+ WARNING:e[0m center of mass shifts (-cmass) are turned off, but would be large
e[7m*+ WARNING:e[0m   - at least one is more than 20% of search range
 +        -cmass x y z shifts =   -1.111  -38.363  -22.990
 +  shift search range is +/- =   57.459   81.855   81.855
 +                                  1.9%    46.9%    28.1%
++ 3dAllineate: total CPU time = 497.2 sec  Elapsed = 123.6
 +  
++ ***********************************************************
e[7m*+ WARNING:e[0m -cmass was turned off, but might have been needed :(
 +           Please check your results - PLEASE PLEASE PLEASE
++ ***********************************************************
3dTstat -min -prefix rm.epi.min.r01 rm.epi.1.r01+tlrc
++ 3dTstat: AFNI version=AFNI_23.2.11 (Sep 26 2023) [64-bit]
++ Authored by: KR Hammett & RW Cox
e[7m** FATAL ERROR:e[0m Can't open dataset rm.epi.1.r01+tlrc
** Program compile date = Sep 26 2023

Here is my afni_proc.py script

#!/bin/tcsh

afni_proc.py                                                                                                                                            \
           -subj_id                  ATN001v1_ON                                                                                             \
           -copy_anat                ./AFNI_SS_opt/ATN001v1_ON/anatSS.ATN001v1_ON.nii                       \
           -anat_has_skull           no                                                                                                            \
           -anat_follower            anat_w_skull anat ./fMRIdata_DICOMsort/ATN001v1_ON/Nifti/"ATN001v1_ON_T1-MPRAGE-8min_20211223085844_301.nii.gz"                                                                                                        \
           -dsets                    ./fMRIdata_DICOMsort/ATN001v1_ON/Nifti/"ATN001v1_ON_MB3_fMRI-12min_Series0601.nii"                                                                                                                            \
           -blocks                   tshift align tlrc volreg mask blur                                                                    \
                                     scale regress                                                                                                     \
           -radial_correlate_blocks  tcat volreg                                                                                            \
           -tcat_remove_first_trs    2                                                                                                            \
           -align_unifize_epi        local                                                                                                         \
           -align_opts_aea           -cost lpc+ZZ                                                                                            \
                                     -giant_move                                                                                                       \
                                     -check_flip                                                                                                          \
           -tlrc_base                MNI152_2009_template_SSW.nii.gz                                                            \
           -tlrc_NL_warp                                                                                                                               \
           -tlrc_NL_warped_dsets     ./AFNI_SS_opt/ATN001v1_ON/anatQQ.ATN001v1_ON.nii               \
                                     ./AFNI_SS_opt/ATN001v1_ON/anatQQ.ATN001v1_ON.aff12.1D                      \
                                     ./AFNI_SS_opt/ATN001v1_ON/anatQQ.ATN001v1_ON_WARP.nii                    \
           -volreg_align_to          MIN_OUTLIER                                                                                           \
           -volreg_align_e2a                                                                                                                           \                                                                                                     
           -volreg_compute_tsnr      yes                                                                                                         \
           -mask_epi_anat            yes                                                                                                            \
           -blur_size                3.0                                                                                                                   \
           -regress_stim_times       ./VDR_data/events/ATN_001v1_ON_events.txt                                      \
           -regress_stim_types       AM2                                                                                                          \
           -regress_basis            'BLOCK(3,1)'                                                                                                \
           -regress_opts_3dD         -jobs 8                                                                                                      \
           -regress_motion_per_run                                                                                                                \
           -regress_censor_motion    0.3                                                                                                         \
           -regress_censor_outliers  0.05                                                                                                        \
           -regress_3dD_stop                                                                                                                           \
           -regress_reml_exec                                                                                                                          \
           -regress_compute_fitts                                                                                                                      \
           -regress_make_ideal_sum   "sum_ideal.1D"                                                                                    \
           -regress_est_blur_epits                                                                                                                     \
           -regress_est_blur_errts                                                                                                                     \
           -regress_run_clustsim     no                                                                                                              \
           -html_review_style        pythonic                                                                                                       \
           -execute

Howdy-

I notice that that message talks about trying to find "rm.epi.1.r01+tlrc" instead of "rm.epi.1.r01+orig". That is, it looks like your input might already be in standard space somehow?

What is the output of:

nifti_tool -disp_hdr -infiles   ./fMRIdata_DICOMsort/ATN001v1_ON/Nifti/"ATN001v1_ON_MB3_fMRI-12min_Series0601.nii" 

Side comment: many things look quite good in that command, I might juuust update your radial_correlate_blocks to this:

-radial_correlate_blocks  tcat volreg  regress

--pt

Hi there, thank you for your reply! I noticed it was trying to find the tlrc and not the orig as well, but when I looked at the script, I couldn't figure out where I messed up the script to have caused this error...

Here is the output I got:

N-1 header file './fMRIdata_DICOMsort/ATN001v1_ON/Nifti/ATN001v1_ON_MB3_fMRI-12min_Series0601.nii', num_fields = 43

all fields:
  name                offset  nvals  values
  ------------------- ------  -----  ------
  sizeof_hdr             0      1    348
  data_type              4     10
  db_name               14     18
  extents               32      1    0
  session_error         36      1    0
  regular               38      1
  dim_info              39      1    0
  dim                   40      8    4 96 96 51 731 1 1 1
  intent_p1             56      1    0.0
  intent_p2             60      1    0.0
  intent_p3             64      1    0.0
  intent_code           68      1    0
  datatype              70      1    64
  bitpix                72      1    64
  slice_start           74      1    0
  pixdim                76      8    -1.0 2.28125 2.28125 2.5 1.0 1.0 1.0 1.0
  vox_offset           108      1    352.0
  scl_slope            112      1    1.0
  scl_inter            116      1    0.0
  slice_end            120      1    0
  slice_code           122      1    0
  xyzt_units           123      1    0
  cal_max              124      1    0.0
  cal_min              128      1    0.0
  slice_duration       132      1    0.0
  toffset              136      1    0.0
  glmax                140      1    0
  glmin                144      1    0
  descrip              148     80
  aux_file             228     24
  qform_code           252      1    0
  sform_code           254      1    2
  quatern_b            256      1    0.0
  quatern_c            260      1    0.985354
  quatern_d            264      1    0.170519
  qoffset_x            268      1    112.617615
  qoffset_y            272      1    -45.55088
  qoffset_z            276      1    -92.303566
  srow_x               280      4    -2.28125 0.0 0.0 112.617615
  srow_y               296      4    0.0 2.148588 -0.840108 -45.55088
  srow_z               312      4    0.0 0.766598 2.354617 -92.303566
  intent_name          328     16
  magic                344      4    n+1

Howdy-

It isn't anything you did in the script. The issue is in the header information for the input EPI dataset, because the qform_code and sform_code are problematic:

qform_code           252      1    0
sform_code           254      1    2

I guess this file was created by some other software that isn't AFNI... sform_code and qform_code values are integers that encode whether the dataset is in "original"/native space coordinates or in standard space coordinates; having values of 3, 4 or 5 mean it is in standard space, while having 0 or 1 mean it is in original/native space. Unfortunately, the interpretation of 2 is ambiguous---it could be either standard or original---this was left in NIFTI as a legacy of some other software formats (and not ones used in AFNI...). So, because that coding is ambiguous, one has to "tell" AFNI whether that code should be interpreted as original or standard space. This is done with the environment variable:

AFNI_NIFTI_VIEW

    The default view extension used for output when creating AFNI format
    datasets from NIFTI datasets.This variable is only applicable for
    sform and qform codes that do not have clearly defined views
    (sform/qform code = 2). Set to "tlrc" or "orig". See also
    AFNI_DEFAULT_STD_SPACE and AFNI_NIFTI_PRIORITY. Note sform/qform code=5
    can be used for spaces other than MNI or TLRC including MNI_ANAT or D99
    spaces.

However, rather than even worry about this with the environment variable, I think it would be better to remove the ambiguity. This could be done with:

# first copy the dset, because the next command overwrites the existing file:
3dcopy DSET NEW_DSET.nii.gz
# fix the space and av_space (=view) information, here knowing the space is orig:
3drefit -space ORIG -view orig NEW_DSET.nii.gz

After doing that, what is the output of:

nifti_tool -disp_hdr -field qform_code -field sform_code -infiles NEW_DSET.nii.gz
3dinfo -space -av_space NEW_DSET.nii.gz

?

--pt

ps: it is probably also worth checking whether the anatomical has this issue, as well:

nifti_tool -disp_hdr -field qform_code -field sform_code \
    -infiles ./AFNI_SS_opt/ATN001v1_ON/anatSS.ATN001v1_ON.nii

I never would've guessed that this was the issue. Thank you so much for your help!!

Here is the output of nifti_tool on the func image:

N-1 header file 'ATN001v1_ON_MB3_fMRI-12min_Series0601_b.nii', num_fields = 2
  name                offset  nvals  values
  ------------------- ------  -----  ------
  qform_code           252      1    5
  sform_code           254      1    5

Here is the output of 3dinfo on the func image:

ATN001v1_ON_MB3_fMRI-12min_Series0601.nii +tlrc

Here is the output I received from nifti_tool on the anat image:

N-1 header file 'anatSS.ATN001v1_ON.nii', num_fields = 2
  name                offset  nvals  values
  ------------------- ------  -----  ------
  qform_code           252      1    1
  sform_code           254      1    1

Hi-

Sorry, I'm a little confused. What is the dataset giving sform_code = 5 and qform_code = 5 now? That looks like the one that just provided values of 0 and 2, respectively.

Did you try using the copy+3drefit command I suggested, and then can you output the nifti_tool command looking at that new dataset?

It's good the anatomical has values of sform_code = 1 and qform_code = 1.

--pt

Hi,
Sorry, I should've clarified! 'ATN001v1_ON_MB3_fMRI-12min_Series0601_b.nii' is the new data set resulting from 3dcopy and 3drefit. I added the "_b" to differentiate it from the original dataset.

But yes, 'ATN001v1_ON_MB3_fMRI-12min_Series0601_b.nii' is the dataset giving sform_code = 5 and qform_code = 5 now, which is the new data set resulted from the original data set that provided values of 0 and 2.

I hope that made sense, but if not, please let me know!

I see, thanks for clarifying.

The weird thing is that the values after that 3dcopy and 3drefit command should not be 5, but instead be 1 or 0. That is quite confusing.

Could we please try something else then:

# copy the dset in a way that all extensions are purged, 
# making a temporary dset
nifti_tool \
   -rm_ext ALL \
   -prefix __tmp.nii \
   -infiles ATN001v1_ON_T1-MPRAGE-8min_20211223085844_301.nii.gz

# edit NIFTI header items
nifti_tool \
   -mod_hdr \
   -mod_field qform_code 1 \
   -mod_field sform_code 1 \
   -prefix ATN001v1_ON_T1-MPRAGE-8min_20211223085844_301_C.nii.gz \
   -infiles __tmp.nii

?

Then, what is the following on that new dset:

nifti_tool -disp_hdr -field qform_code -field sform_code -infiles ATN001v1_ON_T1-MPRAGE-8min_20211223085844_301_C.nii.gz
3dinfo -space -av_space ATN001v1_ON_T1-MPRAGE-8min_20211223085844_301_C.nii.gz

?

--pt

I tried to run the code you suggested and received these as error messages:

The code:

nifti_tool \
   -rm_ext ALL \
   -prefix __tmp.nii \
   -infiles ATN001v1_ON_T1-MPRAGE-8min_20211223085844_301.nii.gz

The error:

** cannot remove 1 exts from image 'ATN001v1_ON_T1-MPRAGE-8min_20211223085844_301.nii.gz' with only 0

2nd Code:

 nifti_tool \
   -mod_hdr \
   -mod_field qform_code 1 \
   -mod_field sform_code 1 \
   -prefix ATN001v1_ON_T1-MPRAGE-8min_20211223085844_301_C.nii.gz \
   -infiles __tmp.nii

Error message:

** ERROR (nifti_read_n1_hdr): failed to find header file for '__tmp.nii'

I also noticed that I had previously ran 3dcopy and 3drefit on my 12 min series. But, the code you just provided for me to try was for my T1-MPRAGE-8min anat dataset. Should I have ran 3dcopy and 3drefit on the original T1-MPRAGE-8min dataset and not the 12min series functional (EPI) dataset?

Okay so I ended up applying the code you most recently provided, but instead of using it for the T1-MPRAGE-8 min anat dataset, I changed it to the ATN001v1_ON_MB3_fMRI-12min_Series0601_b.nii dataset, this is the output I got using nifti_tool and 3dinfo:

N-1 header file 'ATN001v1_ON_MB3_fMRI-12min_series0601_C.nii', num_fields = 2
  name                offset  nvals  values
  ------------------- ------  -----  ------
  qform_code           252      1    1
  sform_code           254      1    1

Sorry, I got those filenames mixed up initially---I had indeed intended it to be run on the EPI dataset.

That looks like good sform_code and qform_code values to have for raw EPI data. Those reflect being in original/native coordinate space.

--pt

Amazing, thank you so much! I appreciate you taking the time to help me with this issue.