3dTrackID Fatal Signal 11 (SIGSEGV) + FATAL ERROR

eji · October 12, 2020, 12:12pm

Hi,
I successfully ran 3dTrackID PROB with a test subject and also using a for loop with two subjects at a time. I now wish to run it for all my subjects (n = 330) using a parallel nohup command. However, I received the error below (here is one example, which is repeated several times for all subjects). Any ideas why? thanks!

Command I ran:
find SPN01* -type f | nohup parallel ‘dtipath=“/home/eji/projects/spins/preproc/fsl/subjects/w0” && cd {} && cd afni && 3dTrackID -mode PROB -uncert DWUncert+orig. -netrois $dtipath/{}/mri/parc5002dwi.nii.gz -mask $dtipath/{}/MASK.nii.gz -prefix 3dTrackID_prob -dti_in 3dDWItoDT_ -alg_Thresh_FA 0.2 -alg_Nmonte 1000 -alg_Nseed_Vox 5 -alg_Thresh_Frac 0.1 -nifti -dump_rois AFNI -no_indipair_out -overwrite’ ::: * &

Output:

++ Tracking mode: PROB
++ Number of ROIs in netw[0] = 318
++ No refset labeltable for naming things.
++ SEARCHING for vector files with prefix ‘3dDWItoDT_':
FINDING: ‘V1’ ‘V2’ ‘V3’
++ SEARCHING for scalar files with prefix '3dDWItoDT_’:
FINDING: not:‘DT’ ‘FA’ ‘L1’ not:‘L2’ not:‘L3’ ‘MD’ ‘RD’
++ Done with scalar search, found: 4 parameters
→ so will have 15 output data matrices.
++ Effective Monte iterations: 5000. Fraction threshold set: 0.10000
→ Ntrack voxel threshold: 500.
++ Tracking progress count: start …

Fatal Signal 11 (SIGSEGV) received
Bottom of Debug Stack
** AFNI version = AFNI_20.1.06 Compile date = May 4 2020
** [[Precompiled binary linux_ubuntu_16_64: May 4 2020]]
** Program Death **
** If you report this crash to the AFNI message board,
** please copy the error messages EXACTLY, and give
** the command line you used to run the program, and
** any other information needed to repeat the problem.
** You may later be asked to upload data to help debug.
** Crash log is appended to file /home/eji/.afni.crashlog

++ Tracking mode: PROB
+ WARNING: If you are performing spatial transformations on an oblique dset,
such as /home/eji/projects/spins/preproc/fsl/subjects/w0/SPN01_MRC_0009_01/mri/parc5002dwi.nii.gz,
or viewing/combining it with volumes of differing obliquity,
you should consider running:
3dWarp -deoblique
on this and other oblique datasets in the same session.
See 3dWarp -help for details.
++ Oblique dataset:/home/eji/projects/spins/preproc/fsl/subjects/w0/SPN01_MRC_0009_01/mri/parc5002dwi.nii.gz is 14.799997 degrees from plumb.
++ Number of ROIs in netw[0] = 318
++ No refset labeltable for naming things.
++ Oblique dataset:/home/eji/projects/spins/preproc/fsl/subjects/w0/SPN01_MRC_0009_01/MASK.nii.gz is 14.799997 degrees from plumb.
++ SEARCHING for vector files with prefix '3dDWItoDT_':
FINDING: ‘V1’ ‘V2’ ‘V3’
++ SEARCHING for scalar files with prefix ‘3dDWItoDT_*’:
FINDING: not:‘DT’ ‘FA’ ‘L1’ not:‘L2’ not:‘L3’ ‘MD’ ‘RD’
++ Done with scalar search, found: 4 parameters
→ so will have 15 output data matrices.
** FATAL ERROR: Input datasets do not match in number of voxels!
‘-netrois’ volume has 1081344 vox,
‘-uncert’ volume has 1750656 vox,
‘-inset’ volume has 1750656 vox,
** Program compile date = May 4 2020

ptaylor · October 12, 2020, 1:49pm

Howdy-

This seems to point to the main problem:


** FATAL ERROR: Input datasets do not match in number of voxels!
'-netrois' volume has 1081344 vox,
'-uncert' volume has 1750656 vox,
'-inset' volume has 1750656 vox,
** Program compile date = May 4 2020

Namely, in the input datasets don’t have the same grid.

A secondary problem might be having obliquity in the data-- probably that is only a problem if there there is differing obliquity across the datasets (in a way that things don’t align properly).

These are the main input files:


DWUncert+orig. 
$dtipath/{}/mri/parc5002dwi.nii.gz
$dtipath/{}/MASK.nii.gz 
3dDWItoDT_*

… and it looks like the network of ROIs one is on the different grid: $dtipath/{}/mri/parc5002dwi.nii.gz.

Can I ask what processing went into the main files (3dDWItoDT_*, DWUncert+orig and $dtipath/{}/MASK.nii.gz), and what wen into $dtipath/{}/mri/parc5002dwi.nii.gz?

Also, what is the output of


3dinfo \
   -obliquity \
   -same_grid \
   -prefix \
   DWUncert+orig.  \
   $dtipath/{}/mri/parc5002dwi.nii.gz  \
   $dtipath/{}/MASK.nii.gz   \
   3dDWItoDT_*

?

–pt

eji · October 21, 2020, 10:44am

Thanks pt, here are the processing steps involved for the files you asked about:

3dDWItoDT_*

3dDWItoDT
-prefix 3dDWItoDT.nii.gz
-automask
-reweight
-max_iter 10
-max_iter_rw 10
-eigs
-sep_dsets
-bmatrix_FULL dwi_matA.txt
dwi_ec_allin.nii.gz
-scale_out_1000
-overwrite

DWUncert+orig

3dDWUncert
-inset dwi_ec_allin.nii.gz
-prefix DWUncert
-input 3dDWItoDT_
-bmatrix_FULL dwi_matA.txt
-iters 50
-overwrite

$dtipath/{}/MASK.nii.gz
To be honest I can’t remember exactly what I did but it is a binary mask. I have attached a screenshot of what it looks like. (is masking even recommended?)
$dtipath/{}/mri/parc5002dwi.nii.gz
Making this netrois file required a number of steps in Freesurfer. The original parcellation (two separate .annot files, one for each hemisphere) is based off the fsaverage_sym template.
-I ran mri_surf2surf to resample from fsaverage_sym onto each subject.
-I then combined lh .annot and rh .annot into a bilateral .nii.gz using mri_aparc2aseg.
-As the parcellation should only be cortical, but mri_aparc2aseg automatically labels subcortical as well, I ran a matlab script to remove the subcortical labels.
-I ran bbregister to get the transformation matrix from dti into T1 space.
-I then used the inverse transformation matrix to get the parcellation (which is in T1 space) to dti space.
-I confirm that this final netrois file is now in dti space and contains 318 parcels.
I hope this wasn’t too confusing.

Also, you asked what the output of 3d info is. Not sure what this output indicates but hope it’s informative:
NO-DSET NO-DSET NO-DSET
0.000 0 parc5002dwi.nii.gz
0.000 0 MASK.nii.gz
NO-DSET NO-DSET NO-DSET

Thanks for your feedback on what might have gone awry.

ptaylor · October 22, 2020, 2:54pm

Thanks for sending that info.

The commands in #1 and #2 look fine. Depending on the data, in the 3dDWUncert command having more than 50 iterations might be useful, like having ~300 hundred. The Bootcamp demo uses 50 for speed considerations. It is a question of convergence of the uncertainty distributions (at the cost of speed for more iterations). But that shouldn’t be causing the issues here in 3dTrackID.

Re. #3:
The mask looks like a mask, though I can’t see it’s overlap with the data… What is “3dinfo -history $dtipath/{}/MASK.nii.gz”?

Re. #4:
Interesting, OK. I am not familiar with this process at all with FS; I normally use the aparc+aseg and aparc+aseg2009 ROIs from their outputs. But given the number of steps and mapping, I wonder if the final grid of this dset is mismatched with the others? What did you use for the mapping/alignment process? AFNI tools, and if so, which one(s)? When bringing them to the diffusion space, is the “-master …” dset one of the diffusion ones, so the grid will be correct?

Re. the 3dinfo command:
Well, I was trying to use your file names and paths… but you need to provide the correct name/path for each of your datasets, which appear to be in different places. The paths for 3dDWUncert+orig and 3dDWItoDT_* are not correct, hence the output of NO-DSET for those two items. Can you please run this modified/expanded form, putting in the correct paths for where “[some path]” is written (if you get NO-DSET showing up, then something isn’t correct and the path needs to be fixed):


3dinfo \
   -obliquity \
   -same_grid \
   -n4   \
   -ad3 \
   -prefix \
   [some path]/DWUncert+orig.  \
   $dtipath/{}/mri/parc5002dwi.nii.gz  \
   $dtipath/{}/MASK.nii.gz   \
   [some path]/3dDWItoDT_DT*

(though I think the output of even that first command shows that the MASK and parc5002 dsets are on different grids.)

-pt

eji · October 26, 2020, 7:04pm

Hi pt, replying to your questions and suggestions:

The output of 3dinfo -history $dtipath/SPN01_CMH_0007_01/MASK.nii.gz is simply “NA”.
Did you ask me to run this because perhaps we could see how MASK.nii.gz was created? I’ll double check with the other researcher who I believe created them. However, I confirm that each binary mask overlays perfectly with each subjects b0 volume (I’ve checked several subjects to be sure).
Re how the mapping/alignment was done: everything was with freesurfer commands. No AFNI tools although I didn’t look into that yet. You mentioned that you normally use the the standard aparc+aseg and aparc+aseg2009 ROIs. Could you describe how you would align that into diffusion space? I assume I could just follow those steps for our T1 parcellation.
Perhaps this is informative: I have attached 2 screenshots, each showing the overlay with parc5002dwi.nii.gz (the -netrois file). One is in diffusion space and the other is in T1 space. You can see that the first is a perfect match, which makes me feel that parc5002dwi.nii.gz was aligned correctly. I have checked this for multiple subjects.
Ok, I re-ran 3dinfo with the correct paths this time:

3dinfo
-obliquity
-same_grid
-n4
-ad3
-prefix
$dtipath/SPN01_CMH_0007_01/afni/DWUncert+orig.
$dtipath/SPN01_CMH_0007_01/mri/parc5002dwi.nii.gz
$dtipath/SPN01_CMH_0007_01/MASK.nii.gz
$dtipath/SPN01_CMH_0007_01/afni/3dDWItoDT_DT*

And this is the output:
0.000 1 128 128 66 6 2.000000 2.000000 2.000000 DWUncert
0.000 1 128 128 66 1 2.000000 2.000000 2.000000 parc5002dwi.nii.gz
0.000 1 128 128 66 1 2.000000 2.000000 2.000000 MASK.nii.gz
0.000 1 128 128 66 6 2.000000 2.000000 2.000000 3dDWItoDT_DT.nii.gz

Thank you again for your feedback on this!

ptaylor · October 27, 2020, 12:47pm

Howdy-

Re. #1: I guess that means that you used non-AFNI programs to generate the MASK.nii.gz file; AFNI ones tend to store a growing list of the commands used into the header, so you can check back for reference later. Okeydoke.

Re. #2a: After running FreeSurfer (FS), you can run @SUMA_Make_Spec_FS to convert the *.mgz/etc. formatted files into NIFTI and GIFTI that other software use. Here is a brief blurb about integrating FS and AFNI like this:
https://afni.nimh.nih.gov/pub/dist/doc/htmldoc/tutorials/fs/fs_fsprep.html
… where you will also get some useful groupings of parcellation and segmentation files (like, maps of WM, GM, ventricle, CSF and other classes), standardized surfaces meshes (so you can use them for group analysis directly), etc.

Re. #2b: There is a (large) tutorial on processing diffusion data with AFNI, including TORTOISE for distortion correction and FS for parcellation segmentation. It is the FATCAT_DEMO2, which is downloadable with the eponymous AFNI command:
@Install_FATCAT_DEMO2
… where there is a full script (do_01_subj_complete.tcsh) and most of the demo output (some intermediates removed, for file size sake), and here is the online set of pages about it:
https://afni.nimh.nih.gov/pub/dist/doc/htmldoc/tutorials/fatcat_prep/main_toc.html
The online pages have full descriptions of outputs and pictures of things, to help understand the steps.
The specific part about mapping GM ROIs to DWI space from FS output is here:
https://afni.nimh.nih.gov/pub/dist/doc/htmldoc/tutorials/fatcat_prep/Postprocessing_II.html
… basically using the command fat_proc_map_to_dti (probably you would want to use @SUMA_Make_Spec_FS prior to this, so the relevant outputs are there).
… and the part about inflating the mapped ROIs potentially and preparing for tracking is on the subsequent page:
https://afni.nimh.nih.gov/pub/dist/doc/htmldoc/tutorials/fatcat_prep/Postprocessing_III.html
… although I note this is even easier now, because @SUMA_Make_Spec_FS now makes a volume of the subset of FS GM ROIs that are just the ROI-like ones (FS estimates a kind of “generic” GM in a few places dotted around the GM, typically borderline cases, and so those shouldn’t be included in tracking; the 3dcalc command there isn’t necessary; you can use the “RENgmrois*” files).

Re. #3: The alignment does look pretty good, though it can be a bit hard to judge with the ROIs being opaque… I think trying fat_proc_map_to_dti might still be useful. The FATCAT_DEMO2 also includes an example of inflating ROIs in a controlled manner, which might be useful, as well (note how gaps can appear between the ROIs from mapping-- the same thing can happen at the GM-WM boundary, making tracking more difficult).

Re. #4: Great, thanks. So, those 4 datasets have no obliquity (just keeps things simple), and are apparently on the same grid (that is what the 1 means). Sooo, that’s good, but now the question is why 3dTrackID showed that some of the inputs were oblique and did have different numbers of voxels…?

Are you sure that your initial 3dTrackID command was using those files, and that “find” command isn’t leading to other ones (with similar names) being input? Can you run the above 3dTrackID command with an additional option “-echo_edu”, which should spit out the full 3dTrackID command being used, with any paths to files expanded and clearly defined? We want to make sure we are examining the same files…

Also, this should use the same files that were being input into your 3dTrackID command:


find SPN01* -type f | nohup parallel 'dtipath="/home/eji/projects/spins/preproc/fsl/subjects/w0" && cd {} && cd afni && \
3dinfo \
-obliquity \
-same_grid \
-orient \
-n4 \
-ad3 \
-prefix \
DWUncert+orig. \
$dtipath/{}/mri/parc5002dwi.nii.gz \
$dtipath/{}/MASK.nii.gz \
3dDWItoDT_DT*' ::: * &

Note also that having 318 ROIs in your GM map is a lot, and can lead to pretty big memory usage potentially; when you ran your successful test case, is this the input GM “netrois” file that you were using?

–pt

eji · November 10, 2020, 2:35pm

Hi pt, replying a couple weeks later.

I confirm that with the test subject, there were also 318 GM rois and no memory issues. We recently increased our memory. Further, I’ve been meaning to try fat_proc_map_to_dti but haven’t yet.

In the meantime, I ran 3dInfo on all subjects to check whether the datasets are on the same grid within-subject. Most of them are, as in the previous subject I showed you. However, below are two examples of where things appear to be mismatched:

Subject A:
3dinfo \

-obliquity
-same_grid
-n4
-ad3
-prefix
$dtipath/SPN01_MRC_0023_01/afni/DWUncert+orig.
$dtipath/SPN01_MRC_0023_01/mri/parc5002dwi.nii.gz
$dtipath/SPN01_MRC_0023_01/MASK.nii.gz
$dtipath/SPN01_MRC_0023_01/afni/3dDWItoDT_DT*
0.000 0 128 135 81 6 2.000000 2.000000 2.000000 DWUncert
6.600 0 128 128 66 1 2.000000 2.000000 2.000000 parc5002dwi.nii.gz
6.600 0 128 128 66 1 2.000000 2.000000 2.000000 MASK.nii.gz
0.000 1 128 135 81 6 2.000000 2.000000 2.000000 3dDWItoDT_DT.nii.gz

Subject B:
3dinfo \

-obliquity
-same_grid
-n4
-ad3
-prefix
$dtipath/SPN01_MRP_0099_01/afni/DWUncert+orig.
$dtipath/SPN01_MRP_0099_01/mri/parc5002dwi.nii.gz
$dtipath/SPN01_MRP_0099_01/MASK.nii.gz
$dtipath/SPN01_MRP_0099_01/afni/3dDWItoDT_DT*
0.000 0 132 143 100 6 2.000000 2.000000 2.000000 DWUncert
14.750 0 128 128 66 1 2.000000 2.000000 2.000000 parc5002dwi.nii.gz
14.750 0 128 128 66 1 2.000000 2.000000 2.000000 MASK.nii.gz
0.000 1 132 143 100 6 2.000000 2.000000 2.000000 3dDWItoDT_DT.nii.gz

I know the first column refers to degrees from plumb, but I don’t understand what plumb is and if it should ideally always be 0. Further, I assume the second column (same_grid) should contain all 1’s in order to be used for 3dTrackID? And finally, what about the differences in the other columns?

Thank you.
Ellen

ptaylor · November 10, 2020, 3:26pm

Hi, Ellen-

The “-obliquity” scalar value output by 3dinfo tells what the largest angle of rotation there is between your FOV coordinates and the scanner coordinates (the obliquity with which the data was acquired); the most important thing here is that your input dsets for a particular subject have differing obliquity. You also having difference matrix sizes for dsets for a given subject (for your Subject A, “128 135 81” is your matrix size for a couple data sets and " 128 128 66" is the matrix size of other ones). It will be impossible to combine these datasets that on different grids (and possibly not even overlapping, in reality).

I’m a little perplexed about how this occurs for only some and not all subjects. Do you have one processing script that you have used, going back to converting DICOMs?

–pt

eji · November 10, 2020, 4:37pm

Hi pt,
Data was collected at 3 different sites.
Upon looking further I’ve noticed that this pattern varies by site such that within-subject datasets are on the same grid for 2 of the sites, but not for the other.
I will check with the PI regarding how the dicoms were converted.

Is there a way to adjust/change the matrices post-hoc so that I would not have to go back to the dicoms?

Thanks.

ptaylor · November 10, 2020, 4:59pm

Hi-

It would be good to see the whole processing, to have the best idea of how to address this consistently, esp. if multiple sites are being used.

–pt

eji · November 10, 2020, 5:14pm

Ok I’ll investigate. But basically what you’re saying is that if the datasets are on diff grids within-subject, it suggests an error during the conversion of the raw dicom files? It wouldn’t have happened later during the various freesurfer/AFNI steps?

eji · November 10, 2020, 5:34pm

To add to my last message: I do indeed have one processing script for everything. I performed each step in batch/parallel for all subjects from beginning to end, minus the very beginning (conversion of dicoms). That is, the converted niftis, bval, bvec etc were given to me. Which is why I’m wondering if the mismatched matrices/datasets is related to the dicom conversion.

ptaylor · November 10, 2020, 5:50pm

Hi, Ellen-

I am not saying that the DICOM->usable dataset conversion is where things went awry, but somewhere there is a problem such that datasets don’t all wind up on the same grid. It would be great to see your processing script, either here or, if you prefer, via email?

–pt

eji · November 10, 2020, 9:11pm

That would be fantastic! I’ve pasted it below. Hopefully it makes sense.
Steps 1-5 should theoretically be the equivalent of fat_proc_map_to_dti I think.

################################################################################

set path for Freesurfer data

export SUBJECTS_DIR=“/home/eji/projects/spins/preproc/freesurfer/subjects/w0”

set path for diffusion data

dtipath=“/home/eji/projects/spins/preproc/fsl/subjects/w0”

------------------------------------------------------------------------------

1) Transpose fsaverage_sym parcellation to each subject (Freesurfer)

------------------------------------------------------------------------------

The fsaverage_sym parcellation has 318 ROIs

cd $SUBJECTS_DIR

for subj in SPN01*; do

left hemisphere

        mri_surf2surf \
            --srcsubject fsaverage_sym \
            --trgsubject $subj \
            --hemi lh \
            --sval-annot fsaverage_sym/label/lh.500_sym.aparc.annot \
            --tval $subj/label/lh.500_sym.aparc.annot;

right hemisphere

        mri_surf2surf \
            --srcsubject fsaverage_sym \
            --trgsubject $subj \
            --hemi rh \
            --sval-annot fsaverage_sym/label/rh.500_sym.aparc.annot \
            --tval $subj/label/rh.500_sym.aparc.annot; done

------------------------------------------------------------------------------

2) Convert left and right hemi parcellation to bilateral nifti (Freesurfer)

------------------------------------------------------------------------------

cd $SUBJECTS_DIR

for subj in SPN01*; do
mri_aparc2aseg
–s $subj
–annot 500_sym.aparc
–wmparc-dmax 2
–labelwm
–hypo-as-wm
–o $subj/mri/aparc.500+2mm.nii.gz; done

------------------------------------------------------------------------------

3) Remove subcortical labels from parcellation (Matlab)

------------------------------------------------------------------------------

cd ~/projects/spins/preproc/freesurfer/scripts

for subj in SPN01*; do
matlab
-nodisplay -r
“renumDesikan_onlycortical(‘/home/eji/projects/spins/preproc/freesurfer/subjects/w0/$subj/mri/aparc.500+2mm.nii.gz’)”; done

------------------------------------------------------------------------------

4) transform b0 into T1 space to get transformation matrix (Freesurfer)

------------------------------------------------------------------------------

cd $dtipath

for subj in SPN01*; do
cd $subj;

bbregister
–s $subj
–mov $dtipath/$subj/data.nii.gz
–dti
–reg mri/register.dat;
cd …/; done

------------------------------------------------------------------------------

5) Transform parcellation from T1 to diffusion space using inverse transformation matrix (Freesurfer)

------------------------------------------------------------------------------

cd $dtipath

for subj in SPN01*; do
cd $subj;

        mri_vol2vol \
            --mov $dtipath/$subj/data.nii.gz \
            --targ $SUBJECTS_DIR/$subj/mri/aparc.500+2mm_ellen.nii.gz \
            --reg mri/register.dat \
            --inv \
            --nearest \
            --o mri/parc5002dwi.nii.gz;

cd …/; done

------------------------------------------------------------------------------

6) Check for gradient flip and use recommended flip for gradient matrices (AFNI)

------------------------------------------------------------------------------

cd $dtipath

for subj in SPN01*; do
cd $subj;
mkdir afni;
cd afni;

@GradFlipTest
-in_dwi …/data.nii.gz
-in_col_vec …/bvec.bvec
-prefix GradFlipTest_rec.txt;

echo the flip value into a file

set my_flip = cat GradFlipTest_rec.txt;

apply recommended @GradFlipTest when converting matrices

1dDW_Grad_o_Mat++
-in_col_vec …/bvec.bvec
-in_bvals …/bval.bval
-out_col_matA dwi_matA.txt
$my_flip;
cd $dtipath/; done

------------------------------------------------------------------------------

7) Transform oblique dataset to cardinal dataset (AFNI)

------------------------------------------------------------------------------

cd $dtipath

for subj in SPN01*; do
cd $subj/afni;

3dWarp
-deoblique
-prefix dwi_ec_deob.nii.gz …/data.nii.gz
-overwrite;
cd $dtipath/; done

------------------------------------------------------------------------------

8) 3dAllineate (AFNI)

------------------------------------------------------------------------------

cd $dtipath

for subj in SPN01*; do
cd $subj/afni;

3dAllineate
-base dwi_ec_deob.nii.gz[0]
-input dwi_ec_deob.nii.gz
-prefix dwi_ec_allin.nii.gz
-cost mutualinfo
-verb
-EPI
cd $dtipath/; done

------------------------------------------------------------------------------

9) Compute tensors and parameters from dti image (AFNI)

------------------------------------------------------------------------------

cd $dtipath

for subj in SPN01*; do
cd $subj/afni;

3dDWItoDT
-prefix 3dDWItoDT.nii.gz
-automask
-reweight
-max_iter 10
-max_iter_rw 10
-eigs
-sep_dsets
-bmatrix_FULL dwi_matA.txt
dwi_ec_allin.nii.gz
-scale_out_1000
-overwrite
cd $dtipath/; done

------------------------------------------------------------------------------

10) Estimate uncertainty of DTI parameters (AFNI)

------------------------------------------------------------------------------

cd $dtipath

for subj in SPN01*; do
cd $subj/afni;

3dDWUncert
-inset dwi_ec_allin.nii.gz
-prefix DWUncert
-input 3dDWItoDT_
-bmatrix_FULL dwi_matA.txt
-iters 50
-overwrite;

cd $dtipath/; done

------------------------------------------------------------------------------

11) Check that within-subject datasets are on same grid (AFNI)

------------------------------------------------------------------------------

pt suggestion Oct 2020

for subj in SPN01*; do
3dinfo
-obliquity
-same_grid
-n4
-ad3
-prefix
$dtipath/$subj/afni/DWUncert+orig.
$dtipath/$subj/mri/parc5002dwi.nii.gz
$dtipath/$subj/MASK.nii.gz
$dtipath/$subj/afni/3dDWItoDT_DT*;
cd $dtipath/; done

------------------------------------------------------------------------------

12) Probabilistic tractography (AFNI)

------------------------------------------------------------------------------

cd $dtipath

for subj in SPN01*; do
cd $subj/afni;

3dTrackID
-mode PROB
-uncert DWUncert+orig.
-netrois $dtipath/$subj/mri/parc5002dwi.nii.gz
-mask $dtipath/$subj/MASK.nii.gz
-prefix 3dTrackID_prob
-dti_in 3dDWItoDT_
-alg_Thresh_FA 0.2
-alg_Nmonte 1000
-alg_Nseed_Vox 5
-alg_Thresh_Frac 0.1
-nifti
-dump_rois AFNI
-no_indipair_out
-overwrite;

cd $dtipath/; done

ptaylor · November 13, 2020, 7:50pm

Hi, Ellen-

A few comments:

A) For FreeSurfer (FS), you presumably ran recon-all and used some option or something to get another parcellation; if AFNI-land, we typically run @SUMA_Make_Spec_FS after running recon-all to bring all the default output into standard mesh GIFTI files and general-purpose NIFTI files; since your extra parcellation appears to be just a *.annot file, you could run @SUMA_Make_Spec_FS as usual, but add this option (see the help file for more options):


-extra_annot_labels L1 L2 ...  : convert extra annot files into ROI dsets
          e.g. -extra_annot_labels aparc
...

For example, if your usual command to translate FS output was:


@SUMA_Make_Spec_FS                            \
    -NIFTI                                    \
    -fspath FS_OUT_DIR                \
    -sid    subj_001

… then this could maybe be adapted to:


@SUMA_Make_Spec_FS                            \
    -NIFTI                                    \
    -fspath FS_OUT_DIR                \
    -extra_annot_labels     500_sym.aparc     \
    -sid    subj_001

Then, to map all the FS+@SUMA_Make_Spec_FS output to diffusion space, you can follow the FATCAT_DEMO2 example, via something like this:


fat_proc_map_to_dti                                                \
    -source          $path_P_ss/anat_02/SUMA/brain.nii             \
    -followers_NN    $path_P_ss/anat_02/SUMA/aparc*_REN_*.nii.gz   \
    -followers_surf  $path_P_ss/anat_02/SUMA/std.141.*.gii         \
    -followers_ndset $path_P_ss/anat_02/SUMA/std.141.*.niml.dset   \
    -followers_spec  $path_P_ss/anat_02/SUMA/std.141.*.spec        \
    -base            $path_P_ss/dwi_05/dwi_dwi.nii.gz'[0]'         \
    -prefix          $path_P_ss/dwi_05/indt

(found here: https://afni.nimh.nih.gov/pub/dist/doc/htmldoc/tutorials/fatcat_prep/Postprocessing_II.html)
… where you might just have another “-followers_NN …” entry, for your new parcellation. Then your outputs would end up in the DWI space definitely.

B) Is your diffusion data oblique? If so, I would be a little concerned about the gradient values-- presumably the gradients are in “scanner coords”, which is why you ran “3dWarp -deoblique …” on the diffusion data? That should (I think) keep the volumetric data and gradients consistent, assuming the gradients were written in scanner coords (not in the frame where the obliquity is ignored). I don’t know about how gradients are recorded when the DWI FOV is oblique-- it would be good to check on that. The downside of running “3dWarp -deoblique …” on a volume is that it will slightly smooth it, because the dataset has to be regridded, applying the obliquity information. It may be there is no way to avoid that eventually, but first it would be good to check on the gradient coordinates (are they scanner coords or not).

C) What is this command doing?


3dAllineate \
-base dwi_ec_deob.nii.gz[0] \
-input dwi_ec_deob.nii.gz \
-prefix dwi_ec_allin.nii.gz \
-cost mutualinfo \
-verb \
-EPI

… is that meant to be distortion correction of some kind? Distortion correction should not be done that way. If it for another purpose, please describe it.

D) For 3dDWUncert, probably a larger number of iterations than 50 should be used, to get a better distribution for estimating uncertainty; I might use 300, even though it is a bit slow at times to calculate. This program is written to make use of multiple CPUs on a computer if you have OpenMP available, so it is worth setting OMP_NUM_THREADS in your script (e.g., in tcsh: “setenv OMP_NUM_THREADS = 6”, assuming you have >6 CPUs on your current machine (you could use all threads on your machine, but probably don’t want to).

–pt

eji · November 14, 2020, 8:23am

Hi pt,

The .annot parcellation was actually sent to us by another group, and is based on the fs_average template, which is why I executed a series of commands to transpose the parcellation onto each individual’s surface and then convert to nifti. Am I understanding correctly that your suggestion re @SUMA_Make_Spec_FS should replace mri_aparc2aseg in our pipeline? (to clarify, I never previously used @SUMA_Make_Spec_FS, so I’m trying to figure out what it adds and where it would fit in)

3dAllineate was used to correct for eddy distortions. However, it sounds like it shouldn’t be used for that and I should remove it entirely.

Also, new, relevant information: I got in touch with the people involved in data acquisition. They confirmed that there were obliquity issues for all subjects who were scanned on the Siemens scanners, which are the same set of subjects I had trouble with (where 3dInfo showed mismatched grid). They mentioned: “we have now preprocessed derivatives for both T1w and diffusion that we are happier with” and fixed it “by re-orienting the image and rotating the bvecs using the align.py script from pnlNipype (https://github.com/pnlbwh/pnlNipype/blob/master/scripts/align.py). This was done at the beginning before any further pre-processing”
– This is not something I am familiar with. Does it sound like this should solve my issue of within-subject data being on different grids?

Thank you and have a nice weekend.

eji · November 24, 2020, 4:15pm

Hi pt.
I finally figured out why some of those files were on different grids. When I transformed the freesurfer file to AFNI, I used the transformation matrix based on oblique dwi data rather than the de-obliqued (3dWarp) data. Thus, this should solve my problem.

It seems like you suggest that I should not use 3dAllineate. Ok.

Further, our collaborators sent their pipeline for dwi pre-processing (not in AFNI, though). The first step is the equivalent of 3dWarp. However, I didn’t perform the equivalent of the last 3 steps in AFNI. Would you suggest I could just use their pre-proprocessed data, or should I do this in AFNI or can I just forget them all?

aligning and centering (pnlNipype and ANTS v2.3.4) — deoblique
denoising with dwidenoise (MRtrix3 v3.0.1)
unringing with mrdegibbs (MRtrix3 v3.0.1)
head motion correction with eddy (FSL v.6.0.1) — originally I thought 3dAllineate took care of this but now it seems not.

ptaylor · November 24, 2020, 6:02pm

Hi, Ellen-

Glad you found the root of the different matrix problem.

The pipeline I use and recommend for distortion correction, tensor estimation, and more is the AFNI+TORTOISE+FreeSurfer pipeline described in the FATCAT_DEMO2 here:
https://afni.nimh.nih.gov/pub/dist/doc/htmldoc/tutorials/fatcat_prep/main_toc.html
It goes from start to finish with processing. The TORTOISE tools DIFFPREP and DR_BUDDI perform Gibbs ringing+motion+eddy current distortion correction and EPI distortion correction, respectively. (Performing EPI distortion correction with DR_BUDDI depends on having a dual phase encoded DWI acquisition.)

I am obviously biased, but I think the FATCAT_DEMO2 describes a good road to be on.

Tying with your previous question:
Re. converting FreeSurfer output to NIFTIs and GIFTIs: yes, I would use @SUMA_Make_Spec_FS. That is part of the FATCAT_DEMO2 pipeline. All the outputs from there get brought to diffusion space in the demo pipeline.

I am not familiar with the Nipype align tool, or what it does exactly with diffusion data, unfortunately.

–pt