Re: can't malloc -1437173116 bytes in 3dNwarpApply

Hi-

I split this question to a new thread from here:
https://afni.nimh.nih.gov/afni/community/board/read.php?1,159764,159764#msg-159764
because it is a somewhat separate question, and the thread has wandered further afield.

–pt


Hi,
I have exactly the same problem as you for NMT to the anat space.
This command has also worked for other animals.


'3dNwarpApply', '-overwrite', '-ainterp', 'NN', '-nwarp', '/lastV/anatT1_unifized_affine_general_warped_WARP.nii.gz', '/lastV/anatT1_unifized_masked_aff.aff12.1D', '-iwarp', '-source', '/Atlas_centr.nii.gz', '-master', '/anatT1.nii.gz', '-prefix', '/Atlas_in_anat.nii.gz']'


++ Processing -nwarp 
 +  -- invert max|AA|=296.562225
 +   - init nstep=10 qq=1/2^10=0.000977
 +   - start iterations: normAA=12.668444 inewtfac=0.333000
 +   - iterate 1 nrat=0.015831
 +   - iterate 2 nrat=0.015625
 +   - iterate 3 nrat=0.008611
 +   - iterate 4 nrat=0.005700
 +   - iterate 5 nrat=0.004121
 +   - iterate 6 nrat=0.002091
 +   - iterate 7 nrat=0.001215
 +   - iterate 8 nrat=0.000514
 +   - switching from linear interp
 +   - iterate 9 nrat=0.000185
 +   - iterate 10 nrat=0.000071
 +  -- iteration converged
++ Warping:** ERROR: EDIT_full_copy: can't malloc -2134449508 bytes for new sub-brick 0
 +    Dataset ./InvertedWarp+orig.HEAD

Fatal Signal 11 (SIGSEGV) received
     THD_nwarp_regrid
    IW3D_set_geometry_nwarp_catlist
   IW3D_read_nwarp_catlist
  3dNwarpApply
 Bottom of Debug Stack
** AFNI version = AFNI_19.3.18  Compile date = Dec 27 2019
** [[Precompiled binary linux_ubuntu_16_64: Dec 27 2019]]
** Program Death **
** If you report this crash to the AFNI message board,
** please copy the error messages EXACTLY, and give
** the command line you used to run the program, and
** any other information needed to repeat the problem.
** You may later be asked to upload data to help debug.
** Crash log is appended to file /home/cgarin/.afni.crashlog


I try to apply the inverse of a transfo based on the anat to NMT that has been inspected visually.
Did you managed to fixe this problem?

“2. I ran two stages transformation. First did the nonlinear warp, saved the result. Then ran the affine. The results looked reasonable. However I have to resample twice in this regime.”
Which AFNI program did you use to separately apply the nonlinear warp and the affine (you can’t use 3dNwarpApply for the affine?)?

Thanks a lot,
Clément

@animal_warper should take care of this for you by doing @Align_Centers. It was tested with the NMT and D99 templates and D99 atlases.

Hi Daniel,
Thank you so much for this advice! I tried @animal_warper and it work so much better than my own code!
I also tried to apply the parameters from @animal_warper to afni_proc.py and tlrc_NL_warped_dsets.
The warp2std_nsu.nii.gz is well coregistrated with the template and well skullstriped.
However warp2std_nsu.nii.gz is transformed by 3dcopy in warp2std_nsu+orig.HEAD.
afni_proc.py required warp2std_nsu+tlrc.HEAD. The problem is that it block totally afni_proc.py
Is there an option for afni_proc to call warp2std_nsu+orig.HEAD instate of warp2std_nsu+tlrc.HEAD? or did I have do something wrong before?
I tried to transform warp2std_nsu+orig.HEAD in warp2std_nsu+tlrc.HEAD with 3dcopy before launching afni_proc. It went to the end but with an EPI image not co-registrated at all to the template.
I think that I don’t really understand the impact of +tlrc and +orig
I would really appreciate some help to solve that.
Thank you very much!
Clément

Hi, Clément-

Can you please post your commands, both the @animal_warper and afni_proc.py commands, here?

Also, just to doublecheck, what is the output of this command:


3dinfo -space -prefix DSET

… for both your nsu dataset, and whatever template you are using as a base? Assuming it is the “NMT” template, the result should be “NMT”, nor “ORIG”.

Note: you can input NIFTI files into afni_proc.py (and really into any AFNI programs).

-pt

Hi ptaylor,
Thank you so much for that answer I think now the origin of my problem is now obvious:


3dinfo -space -prefix /home/cgarin/Documents/1_Macaque_MRI/2_Atlas/nmt_1.2/NMT_SS.nii.gz
** AFNI converts NIFTI_datatype=4 (INT16) in file1_Macaque_MRI/2_Atlas/nmt_1.2/NMT_SS.nii.gz to FLOAT32
     Warnings of this type will be muted for this session.
     Set AFNI_NIFTI_TYPE_WARN to YES to see them all, NO to see none.
ORIG	NMT_SS.nii.gz

3dinfo -space -prefix /home/cgarin/Documents/1_Macaque_MRI/3_Nifti5/Oliver2019_02_02/anatT1_warp2std_nsu.nii.gz
ORIG	anatT1_warp2std_nsu.nii.gz

But I really don’t understand why the space has been modified…
and my code for the co-registration


@animal_warper input anatT1.nii.gz -base 1_Macaque_MRI/2_Atlas/nmt_1.2/NMT_SS.nii.gz -atlas 1_Macaque_MRI/2_Atlas/nmt_1.2/F99_WOcerv_NMT.nii.gz -outdir ouput_dir_a_id

afni_proc.py -subj_id Oliver2019_02_02 /
-script 1_Macaque_MRI/8_Func_AFNI/Oliver2019_02_02/proc.Oliver2019_02_02 /
-scr_overwrite -out_dir 1_Macaque_MRI/8_Func_AFNI/Oliver2019_02_02/Oliver2019_02_02.results /
-blocks tshift align tlrc volreg blur mask scale regress -dsets 1_Macaque_MRI/3_Nifti/Oliver2019_02_02/BOLD_restingstate.nii.gz /
-copy_anat 1_Macaque_MRI/3_Nifti5/Oliver2019_02_02/anatT1_ns.nii.gz -anat_has_skull no -tcat_remove_first_trs 10 /
-volreg_align_to MIN_OUTLIER -volreg_align_e2a -volreg_tlrc_warp -align_opts_aea -cost lpa+zz -giant_move -align_epi_strip_method None /
-tlrc_base 1_Macaque_MRI/2_Atlas/nmt_1.2/NMT_SS.nii.gz', '-tlrc_NL_warp /
-tlrc_NL_warped_dsets 1_Macaque_MRI/3_Nifti5/Oliver2019_02_02/anatT1_warp2std_nsu.nii.gz /
1_Macaque_MRI/3_Nifti5/Oliver2019_02_02/anatT1_shft_al2std_mat.aff12.1D /
1_Macaque_MRI/3_Nifti5/Oliver2019_02_02/anatT1_shft_WARP.nii.gz /
-blur_size 2.0 -regress_censor_motion 0.35 -regress_censor_outliers 0.02 -regress_motion_per_run -regress_est_blur_errts /
-regress_est_blur_epits -regress_run_clustsim no -execute

Let me know if everything sounds good for simple resting state acquisition,
again thanks a lot for the help!

Clément

Also, I just re-downloaded the template and the space remain ORIG. Also, afni_proc seems to required tlrc so even if the space is NMT it will continu to crash no? or is it just ORIG that cause the problem?

Hi, Clément-

Hmm, do you have an older version of the NMT template?

If you type:


3dinfo NMT_SS.nii.gz

what is the top information like “Identifier Code” and “template space”? On my computer, for that file (from the MACAQUE_DEMO), it is:


Dataset File:    NMT_SS.nii.gz
Identifier Code: XYZ_tKgHdnPm4yCdc-49MpfWDQ  Creation Date: Wed Jul 10 18:04:03 2019
Template Space:  NMT

Re. the afni_proc.py code, I will take a look in a bit; how are you running it? I see the forward slashes “/” at the end; a normal continuation of line character is a backslash "". Also, I see some commas and quotes interspersed… Is that just from running it in Python or something?

–pt

Where are you downloading the template from?

Hi again,
I downloaded the template at:
https://afni.nimh.nih.gov/NMT


3dinfo NMT_SS.nii.gz

Identifier Code: AFN_ZuEbKDcZ-wbFK6dSTdvAsQ  Creation Date: Wed Apr 26 15:47:32 2017
Template Space:  ORIG
Dataset Type:    Anat Bucket (-abuc)
Byte Order:      LSB_FIRST {assumed} [this CPU native = LSB_FIRST]
Storage Mode:    NIFTI
Storage Space:   86,035,180 (86 million) bytes
Geometry String: "MATRIX(-0.25,0,0,31.75,0,-0.25,0,50.5,0,0,0.25,-27):253,347,245"
Data Axes Tilt:  Plumb

for the code, yes sorry it was launched in python and I added manually the / and forgot to delete the coma

Hi again,
I downloaded the template at:
https://afni.nimh.nih.gov/NMT


3dinfo NMT_SS.nii.gz

Identifier Code: AFN_ZuEbKDcZ-wbFK6dSTdvAsQ  Creation Date: Wed Apr 26 15:47:32 2017
Template Space:  ORIG
Dataset Type:    Anat Bucket (-abuc)
Byte Order:      LSB_FIRST {assumed} [this CPU native = LSB_FIRST]
Storage Mode:    NIFTI
Storage Space:   86,035,180 (86 million) bytes
Geometry String: "MATRIX(-0.25,0,0,31.75,0,-0.25,0,50.5,0,0,0.25,-27):253,347,245"
Data Axes Tilt:  Plumb

for the code, yes sorry it was launched in python and I added manually the / and forgot to delete the coma

Hi again,
I downloaded the template at:
https://afni.nimh.nih.gov/NMT


3dinfo NMT_SS.nii.gz

Identifier Code: AFN_ZuEbKDcZ-wbFK6dSTdvAsQ  Creation Date: Wed Apr 26 15:47:32 2017
Template Space:  ORIG
Dataset Type:    Anat Bucket (-abuc)
Byte Order:      LSB_FIRST {assumed} [this CPU native = LSB_FIRST]
Storage Mode:    NIFTI
Storage Space:   86,035,180 (86 million) bytes
Geometry String: "MATRIX(-0.25,0,0,31.75,0,-0.25,0,50.5,0,0,0.25,-27):253,347,245"
Data Axes Tilt:  Plumb

for the code, yes sorry it was launched in python and I added manually the / and forgot to delete the coma

Hi, Clément-

I’m not sure about the provenance of that version, and I will ping Daniel about updating that, but if you run:


@Install_MACAQUE_DEMO

… that will download the full demo with scripts, and inside the template/ directory is an updated version of the NMT_SS.nii.gz with appropriate header (i.e., “space”) info. To verify, you can run “3dinfo -space …” on it.

Sorry about the hassle.

–pt

Sorry about the confusion here. I’ve tried to update the datasets to be in NMT space. Also there’s a transformed D99 atlas in that NMT space. Adam Messinger is working with us to make a newer 1.3 version soon.

Great thank you so much for that!
So which template do you suggest to use? or is there a way to bypass this issu directly in afni_proc and work with the ORIG space?
Maybe I don’t understand afni_proc very well, is it totally necessary to have a template in the tlrc space? because I the futur I would like to work with other species and a home made template. Do you think that it will be possible?
Thanks again!
Clément

Hi, Clément-

You can change the header information for the dset in question, to correc the space from being “ORIG” to being “NMT”, with:


3drefit -space NMT warp2std_nsu.nii.gz

That should adjust all related properties of the dset (qform and sform, for example)


nifti_tool -disp_hdr -infiles EEE.nii | grep qform

… and the last column that is output should be a ‘5’ (which means the template is technically classified as ‘NIFTI_XFORM_TEMPLATE_OTHER’; you can read more about the fascinating discussion of this variable and recent expansions to its set of values here:
https://www.nitrc.org/forum/forum.php?thread_id=10029&forum_id=1942)

I believe the expectation is that every reference base will be something that has non-ORIG space value; afni_proc.py has to do work to predict the output names, and this seems a safe assumption. A reference “space” name can be assigned as above; however, probably some caution+circumspection should be used in applying this haphazardly (for interpretation’s sake). In this present case, having an NMT template version with “ORIG” space is a mistake/relic of another era, and so using 3drefit to fix the header makes sense. For future work, using the ‘modern’ NMT_SS.nii.gz file that does have a space name will be highly preferable for pipeline flows.

We have been doing a lot of work with non-human templates (and “we” means Daniel Glen, basically), and non-human templates should still have non-ORIG space values. Actually, it is basically because of such a proliferation of non-MNI and non-Talairach templates that the new [qs]form value of ‘5’ was created. If a given template of interest doesn’t have a space name, please let us know; we would like to start helping people standardize/utilize these kinds of space names.

–pt

Thank you again!
With pleasure! will let you know when I will start my work on other species.

The 3drefit on warp2std_nsu.nii.gz did a perfect job, to change the space and it has allow afni_proc to continu.
However, when it arrived to


# apply catenated xform: volreg/epi2anat/tlrc/NLtlrc
    # then apply non-linear standard-space warp
    3dNwarpApply -master anatT1_warp2std_nsu+tlrc -dxyz 2                 \
                 -source pb01.$subj.r$run.tshift+orig                     \
                 -nwarp "anatT1_shft_WARP.nii.gz mat.r$run.warp.aff12.1D" \
                 -prefix rm.epi.nomask.r$run

rm.epi.nomask output was not co-registrated at all and I can only see 1/4 of the epi image

Do you think that 3drefit might have cause this issu? Or is it something else?

I was thinking trying to start again @animal_warper but with NMT_SS.nii.gz in the good space.
Thanks
Clément

Hi, Clément-

I think the problem is that all the standard space dsets that you would enter into afni_proc.py from warping would have to be updated with the new space value (i.e., all the ones entered with “-tlrc_NL_warped_dsets …”). But in order to be more sure, I think I’d have to ask for you to upload the data for me to really see what is going on… That is possible, but I think it might be more consistent and less frustrating to re-run @animal_warper with the newer NMT_SS.nii.gz, rather than chase through these things.

Let me know what you prefer.

–pt

I think also that sharing this images could be the easiest solution. Thank you so much for your help!
Clément

OK, I have sent PM instructions.

(In the meantime, it might make sense to re-run @animal_warper, too.)

–pt

Hi, Clément-

OK, I have looked at the data and I have a few thoughts:

  • the original coordinates are a bit odd-- while the anatomical and EPI overlay each other, the (x, y, z) = (0, 0, 0) location is far outside the brain; this can affect some alignment issues.
  • the EPI has fairly low contrast (not a lot of details easily seen).
  • both datasets were acquired with oblique coordinates.

So, my thoughts on this:

  • purge the obliquity information, rather than applying it for alignment, because for the anatomical-template alignment, applying the obliquity information will introduce a larger relative rotation to the template brain. Once we purge the anatomical obliquity info, then we should do the same to the EPI (so there isn’t a relative rotation introduced).
  • we should put have (0, 0, 0) of each dset be somewhere in the brain-- mainly to help the anatomical-template alignment.
  • align_epi_anat.py will try to remove the skull of both the anatomical and the EPI by default; after @animal_warper, we will have a skullstripped version of the anatomical, so we should tell align_epi_anat.py that “-anat_has_skull no”; for this EPI, align_epi_anat.py tended to remove a lot of actual brain if it tried to remove the skull (I think because of the general lack of contrast); therefore, one can either not remove the skull of the EPI, or tell it to use “3dAutomask” to do so, via “-epi_strip 3dAutomask”.

Note that when I ran/tested this here, I just used the first 5 time points or so from the EPI time series, for the sake of speed of running.


#!/bin/tcsh

set dset_anat = anatT1.nii.gz
set dset_bold = BOLD_restingstate.nii.gz

set dset_anat_deob = a00_deob.nii
set dset_anat_ss   = a01_ss.nii

set dset_bold_deob = e00_deob.nii

set refvol = template/NMT_SS.nii.gz
set refatl = template/D99_atlas_1.2a_al2NMT.nii.gz
set aw_dir = s01_anwarp

# ==========================================================================

# copy and get rid of obliquity info of anatomical
3dcopy                                \
    ${dset_anat}                      \
    ${dset_anat_deob}

3drefit                               \
    -deoblique                        \
    ${dset_anat_deob}

# ------------------- for center of mass of anatomical
# this should be good enough to center mass around 0 0 0 for template
# alignment
3dSkullStrip                          \
    -prefix    ${dset_anat_ss}        \
    -input     ${dset_anat_deob}      \
    -blur_fwhm 2                      \
    -orig_vol

3dCM                                   \
    -set 0 0 0                         \
    ${dset_anat_ss}

3drefit                                \
    -duporigin ${dset_anat_ss}         \
    ${dset_anat_deob}


# --------------------- for obliquity and center of mass of EPI

3dcalc \
    -a       ${dset_bold}        \
    -expr    'a'                         \
    -prefix  ${dset_bold_deob}

# purge obliquity info, and apply shifts so BOLD dset overlays anat dset well
3drefit                               \
    -deoblique                        \
    ${dset_bold_deob}

3dCM                                  \
    -set 0 0 0                        \
    ${dset_bold_deob}

# ---------------------------------- run main prog for nonlinear warp + skullstripping

@animal_warper                          \
    -input  ${dset_anat_deob}           \
    -base   ${refvol}                   \
    -atlas  ${refatl}                   \
    -outdir ${aw_dir}                   \
    -ok_to_exist


# -----------------------------------------------------------------

# use skullstripped anatomical in orig space (and don't skullstrip it again!);  either use 3dAutomask to skullstrip EPI, or do none at all; results are quite similar:

align_epi_anat.py                         \
    -epi2anat                             \
    -epi ${dset_bold_deob}                \
    -anat ${aw_dir}/a00_deob_ns.nii.gz    \
    -anat_has_skull no \
    -epi_strip 3dAutomask \
    -epi_base 0 -volreg off -tshift off   \
    -suffix TRY5                          \
    -cost lpc+ZZ

align_epi_anat.py                         \
    -epi2anat                             \
    -epi ${dset_bold_deob}                \
    -anat ${aw_dir}/a00_deob_ns.nii.gz    \
    -anat_has_skull no \
    -epi_strip None \
    -epi_base 0 -volreg off -tshift off   \
    -suffix TRY6                          \
    -cost lpc+ZZ

echo "++ Done with aligning anatomical with template"

To provide the same kind of info used in the align_epi_anat.py commands to afni_proc.py, you would use something like the following (exact paths and file names might be different):


-copy_anat      ${aw_dir}/a00_deob_ns.nii.gz        \
-anat_has_skull no                                         \
...
-align_opts_aea                                            \
-cost lpc+ZZ                  \
-epi_strip None            \

… or the same iwth “-epi_strip 3dAutomask”, instead.

Happy to answer any followup questions.

–pt