alignment in afni_proc.py

wgraves · March 11, 2021, 11:36pm

Hello AFNI experts,
As much as I dread asking for help in public, this issue has me stumped. I’m trying to follow afni_proc.py example 11 to pre-process some resting state data. Following that example, I first ran @SSwarper to calculate the transform of the anatomy to MNI space. Then I provided those images to afni_proc.py, along with -align_opts_aea that I separately verified to properly align the anatomy to the EPI images. Running the script afni_proc.py creates does indeed properly align the anatomy to the EPI, as verified by overlaying pb02.subjnum.r01.tshift+orig onto anat_al_junk+orig. However, when I check for alignment of the EPI to MNI space, it’s clearly off (shown by using anatQQ.subjnum+tlrc as the underlay and pb03.subjnum.r01.volreg+tlrc as the overlay). I’ll paste the actual afni_proc.py command next. And of course I’d appreciate any direction you can give me on this!

@SSwarper -input anat.nii
-subid $subj
-odir anat_warped
-base /usr/lib/afni/bin/MNI152_2009_template_SSW.nii.gz

Then:
afni_proc.py
-subj_id $subj
-dsets rest+orig
-blocks despike tshift align tlrc volreg blur mask scale regress
-copy_anat ./anat.nii
-anat_has_skull yes
-anat_follower anat_w_skull anat FT_SurfVol.nii
-anat_follower_ROI aaseg anat aparc.a2009s+aseg.nii
-anat_follower_ROI aeseg epi aparc.a2009s+aseg.nii
-anat_follower_ROI FSvent epi FT_vent.nii
-anat_follower_ROI FSWe epi FT_WM.nii
-anat_follower_erode FSvent FSWe
-tcat_remove_first_trs 2
-align_opts_aea -epi_strip 3dSkullStrip -skullstrip_opts -blur_fwhm 2 -push_to_edge -cost lpc+ -giant_move -check_flip
-tlrc_base MNI152_2009_template_SSW.nii.gz
-tlrc_NL_warp
-tlrc_NL_warped_dsets anatQQ.$subj.nii anatQQ.$subj.aff12.1D anatQQ.${subj}_WARP.nii
-volreg_align_to MIN_OUTLIER
-volreg_align_e2a
-volreg_tlrc_warp
-blur_size 4
-mask_epi_anat yes
-regress_censor_outliers 0.1
-regress_bandpass 0.01 0.1
-regress_apply_mot_types demean deriv
-regress_est_blur_epits
-regress_est_blur_errts
-scr_overwrite
-html_review_style pythonic

Incidentally, the QC webpage output doesn’t seem to show any outlines that are useful for checking registration. Not sure if that’s obtained a different way or what.

ptaylor · March 12, 2021, 2:17pm

Hi, Will-

Hmmm, weird. Just to check—what is your “afni -ver”?

Starting at the end, what does this mean: “the QC webpage output doesn’t seem to show any outlines that are useful for checking registration.”? You should have both a ve2a and va2t QC block for judging EPI-to-anatomical and anatomical-to-template alignment, respectively. Each shows the first dset with the edges of the second overlaid. Is that not the case for you?

Re. warping—that looks like the correct set of inputs and things to me. Just to check—did you verify visually that @SSwarper did a good job of alignment?

And one more thing–what is the output of


3dinfo -history anatQQ.$subj.nii

?

Side note; I notice you have ROI followers, esp. from FreeSurfer parcellation. But I see you are blurring your data (because of the “blur” block present). If you are planning on doing ROI-based analysis, like correlation matrices or something using those follower ROIs, then I don’t think you would want to blur the data. That would artificially increase correlation within neighboring ROIs, as signal got spread across borders.
For voxelwise analyses, blurring makes sense. But not for ROI-based ones.

–pt

wgraves · March 12, 2021, 3:57pm

Hi Paul,
Thank you for the prompt and helpful reply. Here’s the output of afni -ver:
Precompiled binary linux_ubuntu_16_64: Mar 3 2021 (Version AFNI_21.0.13 ‘Titus’)

When I open the QC webpage output for this analysis, I do not seem to have the ve2a and va2t QC blocks. (See attached screen shot.)
I did visually check, and indeed @SSwarper appeared to do a good job aligning the individual anatomy scan to the MNI 2009c template.
Here’s the output of 3dinfo -history anatQQ.$subj.nii:
[wgraves@argus: Sun Mar 7 18:39:14 2021] ===================================
[wgraves@argus: Sun Mar 7 18:39:14 2021] === History of inputs to 3dcalc ===
[wgraves@argus: Sun Mar 7 18:39:14 2021] === Input a:
[wgraves@argus: Sun Mar 7 18:39:14 2021] [ptaylor@Eomer: Sun Jul 1 23:37:45 2018] {AFNI_18.1.33:linux_ubuntu_12_64} 3dcopy MNI152_2009_template.nii.gz MNI152_2009_template_SSW.nii.gz
[wgraves@argus: Sun Mar 7 18:39:11 2021] {AFNI_21.0.13:linux_ubuntu_16_64} 3dQwarp -lite -base ‘/usr/lib/afni/bin/MNI152_2009_template_SSW.nii.gz[0]’ -source anat_warped/junk.SSwarper_qT51PD4gApH_AffSS.nii -iniwarp anat_warped/junk.SSwarper_qT51PD4gApH_QQ9_WARP.nii -warpscale 1 -pcl -inilev 10 -wtgaus 4.5 -inedge -pblur -minpatch 11 -prefix anat_warped/anatQQ.29498.nii
[wgraves@argus: Sun Mar 7 18:39:14 2021] === Input b:
[wgraves@argus: Sun Mar 7 18:39:14 2021] [ptaylor@Eomer: Sun Jul 1 23:37:45 2018] {AFNI_18.1.33:linux_ubuntu_12_64} 3dcopy MNI152_2009_template.nii.gz MNI152_2009_template_SSW.nii.gz
[wgraves@argus: Sun Mar 7 18:39:11 2021] {AFNI_21.0.13:linux_ubuntu_16_64} 3dQwarp -lite -base ‘/usr/lib/afni/bin/MNI152_2009_template_SSW.nii.gz[0]’ -source anat_warped/junk.SSwarper_qT51PD4gApH_AffSS.nii -iniwarp anat_warped/junk.SSwarper_qT51PD4gApH_QQ9_WARP.nii -warpscale 1 -pcl -inilev 10 -wtgaus 4.5 -inedge -pblur -minpatch 11 -prefix anat_warped/anatQQ.29498.nii
[wgraves@argus: Sun Mar 7 18:39:13 2021] {AFNI_21.0.13:linux_ubuntu_16_64} 3dmask_tool -dilate_inputs -1 1 -prefix anat_warped/junk.SSwarper_qT51PD4gApH_de3.nii -input anat_warped/anatQQ.29498.nii
[wgraves@argus: Sun Mar 7 18:39:14 2021] ===================================
[wgraves@argus: Sun Mar 7 18:39:14 2021] {AFNI_21.0.13:linux_ubuntu_16_64} 3dcalc -a anat_warped/anatQQ.29498.nii -b anat_warped/junk.SSwarper_qT51PD4gApH_de3.nii -expr ‘a*step(b)’ -prefix anat_warped/anatQQc.29498.nii

And thanks for the pointer about not blurring with ROI analyses. Makes total sense!

Best,
Will

ptaylor · March 12, 2021, 4:24pm

Thaaat lack of ve2a and va2t is odd. And bad.

Could you email the out.review_html text file that should be in your *.results/ directory created by afni_proc.py? And perhaps also the @ss_review_html script, as well?

thanks,
pt

kxd344 · August 26, 2021, 8:30pm

I had the same issue with no alignment (ve2a and va2t) output for example 11. However, I used to do example 11b and I would get the alignment output. The only difference I can currently guess with comes with the html style. 11b does not differ from the default (basic) whereas example 11 has it set to pythonic. Perhaps the pythonic version misses including ve2a and va2t? I am rerunning example 11 now on my own data without html set to pythonic to see if that makes a difference for me.

About the blur. Are you suggesting to simply remove the “blur” under -blocks? Or remove regress_est_blur_epits or regress_est_blur_errts? Is it important to remove the blur if we are doing the Freesurfer anat_follow (or how much might that change the preprocessing validity/results)? Or is it just important depending on further analysis one might do after the preprocessing from afni_proc.py? Just wanted to check in, because from example 11 in afni_proc.py, the example uses the Freesurfer anat_follower and it includes the blur in blocks and the regress_est_blur_epits and regress_est_blur_errts. Does the example need to be altered for better analysis?

ptaylor · August 26, 2021, 8:47pm

Hi-

There is an important alignment difference between Ex. 11 and 11b: the former uses @SSwarper first, and then applies its results, while the latter does not. In general, the former should give better results for alignment.

Could you please post your afni_proc.py command that is 11-like, namely using @SSwarper? I would also like to see the output.* file that is normally logged when you run afni_proc.py’s command script. There is also the out.review_html created in the “results” directory from afni_proc.py processing, which contains the terminal text of running the QC generation. If you email me those two text files (they are probably too long to post here elegantly), I can take a look and report back.

If, for example, something goes wrong wtih @SSwarper, that could affect these QC blocks. I don’t think it should be the use of “pythonic” (which is much preferred). But I would like to see what the actual code errors that are occurring.

Re. blur:
If you are going to do an ROI-based analysis (e.g., averaging time series within ROIs to make a functional correlation matrix), then you should not blur your data. That will artificially smear signals across neighboring ROI borders. To not include blurring/smoothing in your afni_proc.py command, remove the “blur” tag in the list of blocks, as well as any “-blur_* …” options.
Some people might warp FS ROIs for knowing a location/region, not for averaging time series, so it might be reasonable to both have ROI followers and do blurring.
But you bring up a good point—maybe we should mention in the notes of that example that if you are planning to do ROI-based analysis with the ROI followers, then you should not blur the data.

–pt

kxd344 · August 26, 2021, 10:32pm

When I did 11b in the past, I still did SSWarper first, and just then used that (and anat has no skull). Basically I made 11b just like 11, except for anything having to do with the ventricle mask creation type stuff.

All this being said, you are correct that whether it is pythonic or not does not make a difference as I just tried it with basic again and I did not get the correct files in index.html. So this must be a new issue, because data I ran sswarper on even a week or so ago had all the right alignment showing up, but not today for me. I did notice when I ran SSwarper I got a ton of the message “/home/username/anaconda3/conda/bin/complete: is not a directory” like 15ish times in a row near the beginning, before it continued to run. That looked new to me.

Also, is blur something we can apply after the fact (given the removal of the blur block and blur_size). I’m used to hearing that blur is required before analysis (makes sense with voxel analysis). I have also used SPM12 where you smooth with a blur of [7 7 7] typically at the end, and I have also heard that overall blur should be about 1.5x your voxel size (my voxel size might be around 3x3x3 I think… need to double check). So anyways, I now feel a little unclear when smoothing becomes necessary – but regardless I would think/hope we could just apply a blur post-preprocessing as needed for different analysis (e.g. not ROI based)? In which case [7x7x7] or maybe 1.5*3=[5x5x5] – I assume the choice of blur to add at the end would also depend on if any other previous steps applied some blur (like SSwarper for instance?).

Here is my afni_proc.py command based on 11 specifically. I have tried to remove the blur block and -blur_size 5, but kept the regress_est_blur_errts and regress_est_blur_epits because I assume that’s fine even if it has blur in the name? Anyway, it is almost exactly like example 11, except I had to remove “-radial_correlate_blocks tcat volreg” because it kept erroring out by thinking that -radial_correlate_blocks was a part of the -blocks command, and then blur_size was 5 (because I had larger voxels) and I set regress_censor_motion to 0.25 because our subjects move a lot and even at 0.25 I still lose a lot of data… I know functional can be 0.3, but with resting 0.2 is preferred, but 0.2 just was too much. Hopefully 0.25 is still valid.


   # to not blur (if we want to do ROI analysis later) I removed the blur block in between volreg and mask, as well as -blur* options)
   # so removed -blur_size 5 after volreg_tlrc_warp and -mask_epi_anat yes
   # blurring makes sense for voxel wise analysis
  afni_proc.py                                                 \
    -subj_id ${sid}.${naming}                        	       \
    -blocks despike tshift align tlrc volreg mask scale regress  \
    -copy_anat ${ss_path}/anatSS.${sid}.nii                    	       \
    -anat_has_skull no                                         \
    -anat_follower anat_w_skull anat ${ss_path}/anatU.${sid}.nii	       \
    -anat_follower_ROI aaseg anat ${suma_path}/aparc.a2009s+aseg.nii        \
    -anat_follower_ROI aeseg epi ${suma_path}/aparc.a2009s+aseg.nii         \
    -anat_follower_ROI FSvent epi ${suma_path}/fs_ap_latvent.nii.gz         \
    -anat_follower_ROI FSWe epi ${suma_path}/fs_ap_wm.nii.gz                \
    -anat_follower_erode FSvent FSWe 			       \
    -dsets ${ss_path}/${sid}_rest+orig.HEAD                    	       \
    -tcat_remove_first_trs 0                                   \
    -align_opts_aea -cost lpc+ZZ -ginormous_move -check_flip   \
    -tlrc_base ${template}                                     \
    -tlrc_NL_warp                                              \
    -tlrc_NL_warped_dsets ${ss_path}/anatQQ.${sid}.nii ${ss_path}/anatQQ.${sid}.aff12.1D ${ss_path}/anatQQ.${sid}_WARP.nii  \
    -volreg_align_to MIN_OUTLIER                               \
    -volreg_align_e2a                                          \
    -volreg_tlrc_warp                                          \
    -mask_epi_anat yes                                         \
    -regress_motion_per_run                                    \
    -regress_ROI_PC FSvent 3                                   \
    -regress_ROI_PC_per_run FSvent                             \
    -regress_make_corr_vols aeseg FSvent                       \
    -regress_anaticor_fast                                     \
    -regress_anaticor_label FSWe			       \
    -regress_censor_motion 0.25                                \
    -regress_censor_outliers 0.05                              \
    -regress_apply_mot_types demean deriv                      \
    -regress_est_blur_epits                                    \
    -regress_est_blur_errts                                    \
    -html_review_style pythonic

Related question – to get the appropriate Freesurfer files, I run recon-all, which takes nearly 4-5 hours per subject. Is there another way to get the necessary freesurfer files per subject, without having to run all of it and wait for that long that you know of by chance?

I emailed you the files you requested. Please let me know if you do not get them. Thanks!