Hello,
I have an issue with align_epi_anat.py which I think I have traced back to the source of the error. Perhaps it is a syntax error. I ran into issue originally during afni_proc but am able to reproduce using only the align_epi_anat.py command as below:
align_epi_anat.py -anat2epi -anat
\
> Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01+orig \
> -suffix _al_keep \
> -epi vr_base_min_outlier+orig -epi_base 0 \
> -epi_strip 3dAutomask \
> -cost lpc+ZZ \
> -volreg off -tshift off
#++ align_epi_anat version: 1.58
#++ turning off volume registration
#Script is running (command trimmed):
3dAttribute DELTA ./vr_base_min_outlier+orig
#Script is running (command trimmed):
3dAttribute DELTA ./vr_base_min_outlier+orig
#Script is running (command trimmed):
3dAttribute DELTA ./Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01+orig
#++ Multi-cost is lpc+ZZ
#++ Removing all the temporary files
#Script is running:
\rm -f ./__tt_vr_base_min_outlier*
#Script is running:
\rm -f ./__tt_Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01*
#Script is running (command trimmed):
3dcopy ./Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01+orig ./__tt_Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01+orig
++ 3dcopy: AFNI version=AFNI_18.1.24 (Jun 3 2018) [64-bit]
#++ Removing skull from anat data
#Script is running (command trimmed):
3dSkullStrip -orig_vol -input ./__tt_Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01+orig -prefix ./__tt_Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01_ns
*+ WARNING: If you are performing spatial transformations on an oblique dset,
such as ./__tt_Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01+orig.BRIK,
or viewing/combining it with volumes of differing obliquity,
you should consider running:
3dWarp -deoblique
on this and other oblique datasets in the same session.
See 3dWarp -help for details.
++ Oblique dataset:./__tt_Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01+orig.BRIK is 0.619604 degrees from plumb.
#Script is running (command trimmed):
3dinfo ./__tt_Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01_ns+orig | \grep 'Data Axes Tilt:'|\grep 'Oblique'
#++ Dataset /media/sf_share/PriceNSF/SNP_PreProcess_testing/ProcessedData/SNP052/Y1/SYM/proc/SNP052_SYM.results/__tt_Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01_ns+orig is ***oblique****
#Script is running (command trimmed):
3dAttribute DELTA ./__tt_Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01_ns+orig
#++ Spacing for anat to oblique epi alignment is 1.000000
#++ Matching obliquity of anat to epi
#Script is running (command trimmed):
3dWarp -verb -card2oblique ./vr_base_min_outlier+orig -prefix ./__tt_Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01_ns_ob -newgrid 1.000000 ./__tt_Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01_ns+orig | \grep -A 4 '# mat44 Obliquity Transformation ::' > ./__tt_Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01_ns_obla2e_mat.1D
++ 3dWarp: AFNI version=AFNI_18.1.24 (Jun 3 2018) [64-bit]
++ Authored by: RW Cox
++ Deobliquing original dataset before obliquing
++ Combining oblique transformations
# mat44 Obliquity Transformation ::
0.999986 -0.005189 -0.000641 1.850464
0.005217 0.982668 0.185299 1.828946
-0.000332 -0.185300 0.982682 23.719522
#++ using 0th sub-brick because only one found
#Script is running (command trimmed):
3dbucket -prefix ./__tt_vr_base_min_outlier_ts ./vr_base_min_outlier+orig'[0]'
++ 3dbucket: AFNI version=AFNI_18.1.24 (Jun 3 2018) [64-bit]
#++ resampling epi to match anat data
#Script is running (command trimmed):
3dresample -master ./__tt_Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01_ns_ob+orig -prefix ./__tt_vr_base_min_outlier_ts_rs -inset ./__tt_vr_base_min_outlier_ts+orig'' -rmode Cu
*+ WARNING: If you are performing spatial transformations on an oblique dset,
such as ./__tt_vr_base_min_outlier_ts+orig.BRIK,
or viewing/combining it with volumes of differing obliquity,
you should consider running:
3dWarp -deoblique
on this and other oblique datasets in the same session.
See 3dWarp -help for details.
++ Oblique dataset:./__tt_vr_base_min_outlier_ts+orig.BRIK is 10.688647 degrees from plumb.
#++ removing skull or area outside brain
#Script is running (command trimmed):
3dAutomask -apply_prefix ./__tt_vr_base_min_outlier_ts_rs_ns ./__tt_vr_base_min_outlier_ts_rs+orig
++ 3dAutomask: AFNI version=AFNI_18.1.24 (Jun 3 2018) [64-bit]
++ Authored by: Emperor Zhark
++ Loading dataset ./__tt_vr_base_min_outlier_ts_rs+orig
++ Forming automask
+ Fixed clip level = 69296.054688
+ Used gradual clip level = 67411.656250 .. 74444.046875
+ Number voxels above clip level = 1137027
+ Clustering voxels ...
+ Largest cluster has 1135997 voxels
+ Clustering voxels ...
+ Largest cluster has 1135652 voxels
+ Filled 460 voxels in small holes; now have 1136112 voxels
+ Filled 20983 voxels in large holes; now have 1157095 voxels
+ Clustering voxels ...
+ Largest cluster has 1157081 voxels
+ Clustering non-brain voxels ...
+ Clustering voxels ...
+ Largest cluster has 15582919 voxels
+ Mask now has 1157081 voxels
++ 1157081 voxels in the mask [out of 16740000: 6.91%]
++ first 30 x-planes are zero [from R]
++ last 31 x-planes are zero [from L]
++ first 90 y-planes are zero [from A]
++ last 50 y-planes are zero [from P]
++ first 111 z-planes are zero [from I]
++ last 64 z-planes are zero [from S]
++ applying mask to original data
++ Writing masked data
++ CPU time = 0.000000 sec
#++ Computing weight mask
#Script is running (command trimmed):
3dBrickStat -automask -percentile 90.000000 1 90.000000 ./__tt_vr_base_min_outlier_ts_rs_ns+orig
#++ Applying threshold of 158825.625000 on /media/sf_share/PriceNSF/SNP_PreProcess_testing/ProcessedData/SNP052/Y1/SYM/proc/SNP052_SYM.results/__tt_vr_base_min_outlier_ts_rs_ns+orig
#Script is running (command trimmed):
3dcalc -datum float -prefix ./__tt_vr_base_min_outlier_ts_rs_ns_wt -a ./__tt_vr_base_min_outlier_ts_rs_ns+orig -expr 'min(1,(a/158825.625000))'
++ 3dcalc: AFNI version=AFNI_18.1.24 (Jun 3 2018) [64-bit]
++ Authored by: A cast of thousands
#++ Aligning anat data to epi data
#Script is running (command trimmed):
3dAllineate -lpc+ZZ -wtprefix ./__tt_Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01_ns_ob_al_keep_wtal -weight ./__tt_vr_base_min_outlier_ts_rs_ns_wt+orig -source ./__tt_Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01_ns_ob+orig -prefix ./__tt_Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01_ns_ob_temp_al_keep -base ./__tt_vr_base_min_outlier_ts_rs_ns+orig -nocmass -1Dmatrix_save ./Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01_al_keep_e2a_only_mat.aff12.1D -master SOURCE -weight_frac 1.0 -maxrot 6 -maxshf 10 -VERB -warp aff -source_automask+4 -onepass
++ 3dAllineate: AFNI version=AFNI_18.1.24 (Jun 3 2018) [64-bit]
++ Authored by: Zhark the Registrator
++ lpc+ parameters: hel=0.40 mi=0.20 nmi=0.20 crA=0.40 ov=0.40 [to be zeroed at Final iteration]
++ Source dataset: ./__tt_Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01_ns_ob+orig.HEAD
++ Base dataset: ./__tt_vr_base_min_outlier_ts_rs_ns+orig.HEAD
++ Loading datasets
++ 1438972 voxels in -source_automask+4
++ Zero-pad: not needed
++ 1157077 voxels [6.9%] in weight mask
++ Number of points for matching = 1157077
++ Local correlation: blok type = 'RHDD(6.54321)'
++ -lpc+ parameters: hel=0.40 mi=0.20 nmi=0.20 crA=0.40 ov=0.40 [to be zeroed at Final iteration]
++ shift param auto-range: -59.4..59.4 -96.0..96.0 -96.0..96.0
+ Range param#4 [z-angle] = -6.000000 .. 6.000000
+ Range param#5 [x-angle] = -6.000000 .. 6.000000
+ Range param#6 [y-angle] = -6.000000 .. 6.000000
+ Range param#1 [x-shift] = -10.000000 .. 10.000000
+ Range param#2 [y-shift] = -10.000000 .. 10.000000
+ Range param#3 [z-shift] = -10.000000 .. 10.000000
+ 12 free parameters
++ Normalized convergence radius = 0.001000
++ OpenMP thread count = 4
++ ======= Allineation of 1 sub-bricks using Local Pearson Signed + Others =======
++ ========== sub-brick #0 ========== [total CPU to here=0.0 s]
++ *** Fine pass begins ***
+ * Enter alignment setup routine
+ - copying base image
+ - copying source image
+ - copying weight image
+ - using 1157077 points from base image [use_all=0]
+ * Exit alignment setup routine
+ - histogram: source clip 0 .. 0; base clip 0 .. 0
+ - versus source range 0 .. 178673; base range -3345.71 .. 252353
+ 1092812 total points stored in 2056 'RHDD(6.54321)' bloks
+ - Initial cost = 0.421668
+ - Initial fine Parameters = 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 1.0000 1.0000 0.0000 0.0000 0.0000
***************************************************************************************************************************************************************************************
+ - Finalish cost = 0.066380 ; 477 funcs
+ - Set lpc+ parameters back to pure lpc before Final
+ - histogram: source clip 0 .. 0; base clip 0 .. 0
+ - versus source range 0 .. 178673; base range -3345.71 .. 252353
************************************************************************
+ - Final cost = -0.488088 ; 339 funcs
+ Final fine fit Parameters:
x-shift=-1.2801 y-shift= 2.1715 z-shift= 0.4964
z-angle= 0.4249 x-angle=-2.9908 y-angle=-0.6476
x-scale= 0.9838 y-scale= 1.0002 z-scale= 0.9937
y/x-shear=-0.0001 z/x-shear=-0.0032 z/y-shear= 0.0008
+ - Fine net CPU time = 0.0 s
++ Computing output image
++ image warp: parameters = -1.2801 2.1715 0.4964 0.4249 -2.9908 -0.6476 0.9838 1.0002 0.9937 -0.0001 -0.0032 0.0008
++ Wrote -1Dmatrix_save ./Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01_al_keep_e2a_only_mat.aff12.1D
++ 3dAllineate: total CPU time = 0.0 sec Elapsed = 47.9
++ ###########################################################
++ # Please check results visually for alignment quality #
++ ###########################################################
++ # '-autoweight' is recommended when using -lpc or -lpa #
++ # If your results are not good, please try again. #
++ ###########################################################
#Script is running (command trimmed):
cat_matvec -ONELINE ./Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01_al_keep_e2a_only_mat.aff12.1D ./__tt_Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01_ns_obla2e_mat.1D -I > ./Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01_al_keep_mat.aff12.1D
** ERROR: THD_read_dvecmat: can't read matrix+vector from './__tt_Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01_ns_obla2e_mat.1D'
** FATAL ERROR: Can't read matrix from './__tt_Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01_ns_obla2e_mat.1D'
** Program compile date = Jun 3 2018
#++ Combining anat to epi and oblique transformations
#Script is running (command trimmed):
3dAllineate -base ./__tt_vr_base_min_outlier_ts_rs_ns+orig -1Dmatrix_apply ./Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01_al_keep_mat.aff12.1D -prefix ./Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01_al_keep -input ./__tt_Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01_ns+orig -master SOURCE -weight_frac 1.0 -maxrot 6 -maxshf 10 -VERB -warp aff -source_automask+4 -onepass
++ 3dAllineate: AFNI version=AFNI_18.1.24 (Jun 3 2018) [64-bit]
++ Authored by: Zhark the Registrator
** FATAL ERROR: Can't read -1Dmatrix_apply './Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01_al_keep_mat.aff12.1D' :-(
** Program compile date = Jun 3 2018
#++ Creating final output: anat data aligned to epi
# copy is not necessary
#++ Saving history
#Script is running (command trimmed):
3dNotes -h "align_epi_anat.py -anat2epi -anat \
Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01+orig -suffix _al_keep -epi \
vr_base_min_outlier+orig -epi_base 0 -epi_strip 3dAutomask -cost lpc+ZZ \
-volreg off -tshift off" \
./Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01_al_keep+orig
Error: Cannot open dataset
#++ Removing all the temporary files
#Script is running:
\rm -f ./__tt_vr_base_min_outlier*
#Script is running:
\rm -f ./__tt_Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01*
# Finished alignment successfully
The problem appears to happen here:
3dWarp -verb -card2oblique ./vr_base_min_outlier+orig -prefix ./__tt_Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01_ns_ob -newgrid 1.000000 ./__tt_Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01_ns+orig | \grep -A 4 '# mat44 Obliquity Transformation ::' > ./__tt_Price_234469.17.01.17-40-09.WIP_T1_MPRAGE.01_ns_obla2e_mat.1D
++ 3dWarp: AFNI version=AFNI_18.1.24 (Jun 3 2018) [64-bit]
++ Authored by: RW Cox
++ Deobliquing original dataset before obliquing
++ Combining oblique transformations
# mat44 Obliquity Transformation ::
0.999986 -0.005189 -0.000641 1.850464
0.005217 0.982668 0.185299 1.828946
-0.000332 -0.185300 0.982682 23.719522
Because the resulting “…obla2e_mat.1D” file is empty. If I paste that transformation matrix manually into file, it seems to be happy. Is there something wrong with this pipe syntax perhaps? I am running this in Ubuntu 16.04 vm. I can get around the error using -deoblique off, but would be nice to keep that on if possible.
EDIT: After some digging, maybe this is stdio buffering issue. Although, suggestions to fix that (e.g. using “grep --line-buffered” or “stdbuf -o0 grep” such as mentioned here https://www.perkin.org.uk/posts/how-to-fix-stdio-buffering.html) don’t seem to change behavior…
Thanks!
-BC