Issue with scaling during preprocessing

AFNI version info (afni -ver): Aug 22 2023 (Version AFNI_23.2.08 'Marcus Didius Severus Julianus')

code text  # or delete if not needed

# ================================== mask ==================================
# create 'full_mask' dataset (union mask)
foreach run ( $runs )
    3dAutomask -dilate 1 -prefix rm.mask_r$run pb02.$subj.r$run.volreg+orig
end

# get mean and compare it to 0 for taking 'union'
3dMean -datum short -prefix rm.mean rm.mask*.HEAD
3dcalc -a rm.mean+orig -expr 'ispositive(a-0)' -prefix full_mask.$subj


# ================================= empty ==================================
# empty block: use '3dTcat' as a placeholder command

foreach run ( $runs )
    3dTproject -prefix pb03.$subj.r$run.tproject -input pb02.$subj.r$run.volreg+orig -stopband 0 0.008 ## 0.008 better than 0.01 
end

# ================================= scale ==================================
# scale each voxel time series to have a mean of 100
# (be sure no negatives creep in)
# (subject to a range of [0,200])
# earlier: -expr 'min(200, a/b*100)*step(a)*step(b)'
foreach run ( $runs )
    3dTstat -prefix rm.mean_r$run pb03.$subj.r$run.tproject+orig
    3dcalc -a pb03.$subj.r$run.tproject+orig -b rm.mean_r$run+orig \
           -expr 'min(200, a/b*100)'            \
           -prefix pb04.$subj.r$run.scale
end

Hello,

I am trying to scale the output of 3dTproject, but it ends up looking like it has a lot of drop out and it has become noisy. Everything also ends up with value of 200. I am not sure why this is happening and if there is a better way to scale the data.

Here is the image of result from scaling

Hello,

That Tproject is removing the low frequencies, now there is no mean to scale to, so many values are theoretically infinite, and get capped at 200. You might be better off having afni_proc.py run in full, and do it using -regress_bandpass, e.g.

-regress_polort 2         \
-regress_bandpass 0.01 1  \
  • rick

I tried running the afni_proc.py using those option, but it is giving me an error about computation failed. It stops after scaling. I am copying the whole output of the file.

I used this command to get the preprocessing script.
afni_proc.py -subj_id 001
-dsets 001/functs/001_run1+orig.HEAD 001/functs/001_run2+orig.HEAD 001/functs/001_run3+orig.HEAD 001/functs/001_run4+orig.HEAD 001/functs/001_run5+orig.HEAD
-script 001_proc_script
-regress_polort 2
-regress_bandpass 0.01 1

Here is part of the output of the script:

3dmask_tool -inputs rm.mask_r01+orig.HEAD rm.mask_r02+orig.HEAD rm.mask_r03+orig.HEAD rm.mask_r04+orig.HEAD rm.mask_r05+orig.HEAD -union -prefix full_mask.001
++ processing 5 input dataset(s), NN=2...
++ padding all datasets by 0 (for dilations)
e[7m*+ WARNING:e[0m If you are performing spatial transformations on an oblique dset,
such as ./rm.mask_r01+orig.BRIK,
or viewing/combining it with volumes of differing obliquity,
you should consider running:
3dWarp -deoblique
on this and other oblique datasets in the same session.
See 3dWarp -help for details.
++ Oblique dataset:./rm.mask_r01+orig.BRIK is 17.000006 degrees from plumb.
++ Oblique dataset:./rm.mask_r02+orig.BRIK is 17.000006 degrees from plumb.
++ Oblique dataset:./rm.mask_r03+orig.BRIK is 17.000006 degrees from plumb.
++ Oblique dataset:./rm.mask_r04+orig.BRIK is 17.000006 degrees from plumb.
++ Oblique dataset:./rm.mask_r05+orig.BRIK is 17.000006 degrees from plumb.
++ frac 0 over 5 volumes gives min count 0
++ voxel limits: 0 clipped, 41471 survived, 105985 were zero
++ writing result full_mask.001...
foreach run ( 01 02 03 04 05 )
3dTstat -prefix rm.mean_r01 pb03.001.r01.blur+orig
++ 3dTstat: AFNI version=AFNI_23.2.08 (Aug 22 2023) [64-bit]
++ Authored by: KR Hammett & RW Cox
e[7m*+ WARNING:e[0m If you are performing spatial transformations on an oblique dset,
such as ./pb03.001.r01.blur+orig.BRIK,
or viewing/combining it with volumes of differing obliquity,
you should consider running:
3dWarp -deoblique
on this and other oblique datasets in the same session.
See 3dWarp -help for details.
++ Oblique dataset:./pb03.001.r01.blur+orig.BRIK is 17.000006 degrees from plumb.
++ Output dataset ./rm.mean_r01+orig.BRIK
3dcalc -a pb03.001.r01.blur+orig -b rm.mean_r01+orig -expr min(200, a/b*100)step(a)step(b) -prefix pb04.001.r01.scale
++ 3dcalc: AFNI version=AFNI_23.2.08 (Aug 22 2023) [64-bit]
++ Authored by: A cast of thousands
++ Output dataset ./pb04.001.r01.scale+orig.BRIK
end
3dTstat -prefix rm.mean_r02 pb03.001.r02.blur+orig
++ 3dTstat: AFNI version=AFNI_23.2.08 (Aug 22 2023) [64-bit]
++ Authored by: KR Hammett & RW Cox
e[7m
+ WARNING:e[0m If you are performing spatial transformations on an oblique dset,
such as ./pb03.001.r02.blur+orig.BRIK,
or viewing/combining it with volumes of differing obliquity,
you should consider running:
3dWarp -deoblique
on this and other oblique datasets in the same session.
See 3dWarp -help for details.
++ Oblique dataset:./pb03.001.r02.blur+orig.BRIK is 17.000006 degrees from plumb.
++ Output dataset ./rm.mean_r02+orig.BRIK
3dcalc -a pb03.001.r02.blur+orig -b rm.mean_r02+orig -expr min(200, a/b
100)step(a)step(b) -prefix pb04.001.r02.scale
++ 3dcalc: AFNI version=AFNI_23.2.08 (Aug 22 2023) [64-bit]
++ Authored by: A cast of thousands
++ Output dataset ./pb04.001.r02.scale+orig.BRIK
end
3dTstat -prefix rm.mean_r03 pb03.001.r03.blur+orig
++ 3dTstat: AFNI version=AFNI_23.2.08 (Aug 22 2023) [64-bit]
++ Authored by: KR Hammett & RW Cox
e[7m
+ WARNING:e[0m If you are performing spatial transformations on an oblique dset,
such as ./pb03.001.r03.blur+orig.BRIK,
or viewing/combining it with volumes of differing obliquity,
you should consider running:
3dWarp -deoblique
on this and other oblique datasets in the same session.
See 3dWarp -help for details.
++ Oblique dataset:./pb03.001.r03.blur+orig.BRIK is 17.000006 degrees from plumb.
++ Output dataset ./rm.mean_r03+orig.BRIK
3dcalc -a pb03.001.r03.blur+orig -b rm.mean_r03+orig -expr min(200, a/b
100)step(a)step(b) -prefix pb04.001.r03.scale
++ 3dcalc: AFNI version=AFNI_23.2.08 (Aug 22 2023) [64-bit]
++ Authored by: A cast of thousands
++ Output dataset ./pb04.001.r03.scale+orig.BRIK
end
3dTstat -prefix rm.mean_r04 pb03.001.r04.blur+orig
++ 3dTstat: AFNI version=AFNI_23.2.08 (Aug 22 2023) [64-bit]
++ Authored by: KR Hammett & RW Cox
e[7m
+ WARNING:e[0m If you are performing spatial transformations on an oblique dset,
such as ./pb03.001.r04.blur+orig.BRIK,
or viewing/combining it with volumes of differing obliquity,
you should consider running:
3dWarp -deoblique
on this and other oblique datasets in the same session.
See 3dWarp -help for details.
++ Oblique dataset:./pb03.001.r04.blur+orig.BRIK is 17.000006 degrees from plumb.
++ Output dataset ./rm.mean_r04+orig.BRIK
3dcalc -a pb03.001.r04.blur+orig -b rm.mean_r04+orig -expr min(200, a/b
100)step(a)step(b) -prefix pb04.001.r04.scale
++ 3dcalc: AFNI version=AFNI_23.2.08 (Aug 22 2023) [64-bit]
++ Authored by: A cast of thousands
++ Output dataset ./pb04.001.r04.scale+orig.BRIK
end
3dTstat -prefix rm.mean_r05 pb03.001.r05.blur+orig
++ 3dTstat: AFNI version=AFNI_23.2.08 (Aug 22 2023) [64-bit]
++ Authored by: KR Hammett & RW Cox
e[7m
+ WARNING:e[0m If you are performing spatial transformations on an oblique dset,
such as ./pb03.001.r05.blur+orig.BRIK,
or viewing/combining it with volumes of differing obliquity,
you should consider running:
3dWarp -deoblique
on this and other oblique datasets in the same session.
See 3dWarp -help for details.
++ Oblique dataset:./pb03.001.r05.blur+orig.BRIK is 17.000006 degrees from plumb.
++ Output dataset ./rm.mean_r05+orig.BRIK
3dcalc -a pb03.001.r05.blur+orig -b rm.mean_r05+orig -expr min(200, a/b
100)step(a)step(b) -prefix pb04.001.r05.scale
++ 3dcalc: AFNI version=AFNI_23.2.08 (Aug 22 2023) [64-bit]
++ Authored by: A cast of thousands
++ Output dataset ./pb04.001.r05.scale+orig.BRIK
end
1d_tool.py -infile dfile_rall.1D -set_run_lengths 37 91 128 128 160 -demean -write motion_demean.1D
1d_tool.py -infile dfile_rall.1D -set_run_lengths 37 91 128 128 160 -derivative -demean -write motion_deriv.1D
foreach index ( count -digits 1 1 $#runs )
count -digits 1 1 5
set nt = 37
set run = 01
1dBport -nodata 37 2 -band 0.008 1 -invert -nozero
++ Block length: 37
e[7m
FATAL ERROR:e[0m Computation fails! :-(((
** Program compile date = Aug 22 2023

With TR=2s and only 37 time points, that is only 74 seconds (72 really) of time in the data. That bandpass operation is trying to create regressors with cycles of duration 125 s, and it is surely making the program fail. If you only have 74 seconds of data, just use -polort 1 or 2 and be done with it.

  • rick