Hi experts,
I’m trying to process my macaque data with afni_proc.py. I followed the example scripts from @Install_MACAQUE_DEMO_REST.
An error that reads “** FATAL ERROR: -polort value can’t be over 20” occurred when running @radial_correlate( see the attachment). I tried to add a flag "-radial_correlate_opts -polort 18 ", but changed nothing.
Could you please tell me what should I do then?
Best,
Jewel
Hi, Jewel-
That does seem like a high order of polynomials. Is the time interval of that set of EPI data (total number of time points times TR) quite long?
Maybe a maximum polynort number could be specified, and a high pass filter used to remove the baseline in this case.
What is the full afni_proc.py (AP) command, and do you have a the tee’ed log file of terminal text for the complete output?
thanks,
pt
Yes, the scan time is a little bit long. There are 1600 time points with TR=2s. Here is the full command and log. Could you please let me know how to modify it? Or can I split the full timeseries into two timeseries images with 800 time points each and merge them after finishing the preprocessing?
afni_proc.py \
-subj_id ${subj} \
-blocks tshift align tlrc volreg blur mask scale regress \
-dsets ${dsets_epi} \
-copy_anat ${anat_cp} \
-anat_has_skull no \
-anat_uniform_method none \
-radial_correlate_blocks tcat volreg \
-radial_correlate_opts -sphere_rad 14 \
-volreg_align_to MIN_OUTLIER \
-volreg_align_e2a \
-volreg_tlrc_warp \
-volreg_warp_dxyz ${final_dxyz} \
-volreg_compute_tsnr yes \
-align_epi_strip_method 3dSkullStrip \
-tlrc_base ${ref_base} \
-tlrc_NL_warp \
-tlrc_NL_warped_dsets ${dsets_NL_warp} \
-blur_size ${blur_size} \
-regress_motion_per_run \
-regress_apply_mot_types demean deriv \
-regress_censor_motion ${cen_motion} \
-regress_censor_outliers ${cen_outliers} \
-regress_est_blur_errts \
-regress_est_blur_epits \
-regress_run_clustsim no \
-html_review_style pythonic \
log:
++ Proc command: 20_ap_vox
++ Found 1 subj:
++ Prepare cmd for: sub-032163 --- ses-1
++ Executing 'swarm' cmd script: /Macaque/afni_test/swarm/swarm_20_ap_vox.tcsh
set subj = sub-032163
set ses = ses-1
set ap_label = 20_ap_vox
set dir_inroot = /Macaque/afni_test
set dir_log = /Macaque/afni_test/logs
set dir_ref = /Macaque/afni_test/NMT_v2.0_sym/NMT_v2.0_sym_05mm
set dir_basic = /Macaque/afni_test/BIDS
set dir_aw = /Macaque/afni_test/AFNI_aw
set dir_ap = /Macaque/afni_test/AFNI_ap
set sdir_basic = /Macaque/afni_test/BIDS/sub-032163/ses-1
set sdir_anat = /Macaque/afni_test/BIDS/sub-032163/ses-1/anat
set sdir_epi = /Macaque/afni_test/BIDS/sub-032163/ses-1/func
set sdir_aw = /Macaque/afni_test/AFNI_aw/sub-032163/ses-1
set sdir_ap = /Macaque/afni_test/AFNI_ap/sub-032163/ses-1
set anat_orig = /Macaque/afni_test/BIDS/sub-032163/ses-1/anat/sub-032163*T1w.nii.gz
set anat_orig_ab = sub-032163_anat
set ref_base = /Macaque/afni_test/NMT_v2.0_sym/NMT_v2.0_sym_05mm/NMT_v2.0_sym_05mm_SS.nii.gz
set ref_base_ab = NMT2
set ref_atl = ( /Macaque/afni_test/NMT_v2.0_sym/NMT_v2.0_sym_05mm/CHARM_in_NMT_v2.0_sym_05mm.nii.gz /Macaque/afni_test/NMT_v2.0_sym/NMT_v2.0_sym_05mm/D99_atlas_in_NMT_v2.0_sym_05mm.nii.gz )
set ref_atl_ab = ( CHARM D99 )
set ref_seg = /Macaque/afni_test/NMT_v2.0_sym/NMT_v2.0_sym_05mm/NMT_v2.0_sym_05mm_segmentation.nii.gz
set ref_seg_ab = SEG
set ref_mask = /Macaque/afni_test/NMT_v2.0_sym/NMT_v2.0_sym_05mm/NMT_v2.0_sym_05mm_brainmask.nii.gz
set ref_mask_ab = MASK
set dsets_epi = ( /Macaque/afni_test/BIDS/sub-032163/ses-1/func/sub-032163*bold.nii.gz )
set anat_cp = /Macaque/afni_test/AFNI_aw/sub-032163/ses-1/sub-032163_anat_nsu.nii.gz
set dsets_NL_warp = ( /Macaque/afni_test/AFNI_aw/sub-032163/ses-1/sub-032163_anat_warp2std_nsu.nii.gz /Macaque/afni_test/AFNI_aw/sub-032163/ses-1/sub-032163_anat_composite_linear_to_template.1D /Macaque/afni_test/AFNI_aw/sub-032163/ses-1/sub-032163_anat_shft_WARP.nii.gz )
set blur_size = 2.0
set final_dxyz = 1.25
set cen_motion = 0.1
set cen_outliers = 0.02
set cost_a2e = lpc+ZZ
set aea_extra =
set aea_uni_epi = yes
set nthr_avail = `afni_system_check.py -disp_num_cpu`
afni_system_check.py -disp_num_cpu
set nthr_using = `afni_check_omp`
afni_check_omp
echo ++ INFO: Using 12 of available 12 threads
++ INFO: Using 12 of available 12 threads
setenv AFNI_COMPRESSOR GZIP
set ap_cmd = /Macaque/afni_test/AFNI_ap/sub-032163/ses-1/ap.cmd.sub-032163
mkdir -p /Macaque/afni_test/AFNI_ap/sub-032163/ses-1
cat
cd /Macaque/afni_test/AFNI_ap/sub-032163/ses-1
tcsh -xef /Macaque/afni_test/AFNI_ap/sub-032163/ses-1/ap.cmd.sub-032163
tee output.ap.cmd.sub-032163
afni_proc.py -subj_id sub-032163 -blocks tshift align tlrc volreg blur mask scale regress -dsets /Macaque/afni_test/BIDS/sub-032163/ses-1/func/sub-032163_ses-001_task-resting_run-1_bold.nii.gz -copy_anat /Macaque/afni_test/AFNI_aw/sub-032163/ses-1/sub-032163_anat_nsu.nii.gz -anat_has_skull no -anat_uniform_method none -radial_correlate_blocks tcat volreg -radial_correlate_opts -sphere_rad 14 -volreg_align_to MIN_OUTLIER -volreg_align_e2a -volreg_tlrc_warp -volreg_warp_dxyz 1.25 -volreg_compute_tsnr yes -align_epi_strip_method 3dSkullStrip -tlrc_base /Macaque/afni_test/NMT_v2.0_sym/NMT_v2.0_sym_05mm/NMT_v2.0_sym_05mm_SS.nii.gz -tlrc_NL_warp -tlrc_NL_warped_dsets /Macaque/afni_test/AFNI_aw/sub-032163/ses-1/sub-032163_anat_warp2std_nsu.nii.gz /Macaque/afni_test/AFNI_aw/sub-032163/ses-1/sub-032163_anat_composite_linear_to_template.1D /Macaque/afni_test/AFNI_aw/sub-032163/ses-1/sub-032163_anat_shft_WARP.nii.gz -blur_size 2.0 -regress_motion_per_run -regress_apply_mot_types demean deriv -regress_censor_motion 0.1 -regress_censor_outliers 0.02 -regress_est_blur_errts -regress_est_blur_epits -regress_run_clustsim no -html_review_style pythonic
-- applying input view as +orig
-- template = 'NMT_v2.0_sym_05mm_SS.nii.gz', exists = 1
-- will use min outlier volume as motion base
-- including default: -find_var_line_blocks tcat
-- tcat: reps is now 1600
++ updating polort to 22, from run len 3212.0 s
-- importing NL-warp datasets
-- volreg: using base dset vr_base_min_outlier+orig
++ volreg: applying volreg/epi2anat/tlrc xforms to isotropic 1.25 mm tlrc voxels
-- masking: group anat = 'NMT_v2.0_sym_05mm_SS.nii.gz', exists = 1
-- have 1 ROI dict entries ...
-- will apply motion types: demean, deriv
-- using default: will not apply EPI Automask
(see 'MASKING NOTE' from the -help for details)
-------------------------------------
** warning have only 1 run to analyze
-------------------------------------
--> script is file: proc.sub-032163
to execute via tcsh:
tcsh -xef proc.sub-032163 |& tee output.proc.sub-032163
to execute via bash:
tcsh -xef proc.sub-032163 2>&1 | tee output.proc.sub-032163
tcsh -xef proc.sub-032163
tee output.proc.sub-032163
echo auto-generated by afni_proc.py, Fri May 19 15:01:51 2023
auto-generated by afni_proc.py, Fri May 19 15:01:51 2023
echo (version 7.55, March 1, 2023)
(version 7.55, March 1, 2023)
echo execution started: `date`
date
execution started: Fri May 19 15:01:52 CST 2023
afni -ver
Precompiled binary linux_ubuntu_16_64: May 8 2023 (Version AFNI_23.1.05 'Publius Helvius Pertinax')
afni_history -check_date 14 Nov 2022
-- is current: afni_history as new as: 14 Nov 2022
most recent entry is: 05 May 2023
if ( 0 ) then
if ( 0 > 0 ) then
set subj = sub-032163
endif
set output_dir = sub-032163.results
if ( -d sub-032163.results ) then
set runs = ( `count -digits 2 1 1` )
count -digits 2 1 1
mkdir -p sub-032163.results
mkdir sub-032163.results/stimuli
3dcopy /Macaque/afni_test/AFNI_aw/sub-032163/ses-1/sub-032163_anat_nsu.nii.gz sub-032163.results/sub-032163_anat_nsu
++ 3dcopy: AFNI version=AFNI_23.1.05 (May 8 2023) [64-bit]
3dcopy /Macaque/afni_test/NMT_v2.0_sym/NMT_v2.0_sym_05mm/NMT_v2.0_sym_05mm_SS.nii.gz sub-032163.results/NMT_v2.0_sym_05mm_SS.nii.gz
++ 3dcopy: AFNI version=AFNI_23.1.05 (May 8 2023) [64-bit]
3dcopy /Macaque/afni_test/AFNI_aw/sub-032163/ses-1/sub-032163_anat_warp2std_nsu.nii.gz sub-032163.results/sub-032163_anat_warp2std_nsu
++ 3dcopy: AFNI version=AFNI_23.1.05 (May 8 2023) [64-bit]
3dcopy /Macaque/afni_test/AFNI_aw/sub-032163/ses-1/sub-032163_anat_composite_linear_to_template.1D sub-032163.results/sub-032163_anat_composite_linear_to_template.1D
++ 3dcopy: AFNI version=AFNI_23.1.05 (May 8 2023) [64-bit]
3dcopy /Macaque/afni_test/AFNI_aw/sub-032163/ses-1/sub-032163_anat_shft_WARP.nii.gz sub-032163.results/sub-032163_anat_shft_WARP.nii.gz
++ 3dcopy: AFNI version=AFNI_23.1.05 (May 8 2023) [64-bit]
3dTcat -prefix sub-032163.results/pb00.sub-032163.r01.tcat /Macaque/afni_test/BIDS/sub-032163/ses-1/func/sub-032163_ses-001_task-resting_run-1_bold.nii.gz[0..$]
++ 3dTcat: AFNI version=AFNI_23.1.05 (May 8 2023) [64-bit]
++ elapsed time = 19.7 s
set tr_counts = ( 1600 )
cd sub-032163.results
@radial_correlate -nfirst 0 -polort 22 -do_clean yes -rdir radcor.pb00.tcat -sphere_rad 14 pb00.sub-032163.r01.tcat+orig.HEAD
++ 3dTcat: AFNI version=AFNI_23.1.05 (May 8 2023) [64-bit]
++ elapsed time = 23.0 s
++ 3dAutomask: AFNI version=AFNI_23.1.05 (May 8 2023) [64-bit]
++ Authored by: Emperor Zhark
-- detrend -polort 22, new eset = det.r01
++ 3dTproject: AFNI version=AFNI_23.1.05 (May 8 2023) [64-bit]
++ Authored by: Cox the Algebraic (Linear)
** FATAL ERROR: -polort value can't be over 20 :-(
** Program compile date = May 8 2023
-- running correlation on dataset pb00.sub-032163.r01.tcat+orig.HEAD ...
++ merge blur: rad 14, FWHM 28
++ 3dmerge: AFNI version=AFNI_23.1.05 (May 8 2023) [64-bit]
*** cannot open dataset det.r01+orig
Program 3dmerge
3dmerge: edit and combine 3D datasets, by RW Cox
++ 3dTcorrelate: AFNI version=AFNI_23.1.05 (May 8 2023) [64-bit]
** FATAL ERROR: Can't open input dataset sphere.mean.14.r01+orig
** Program compile date = May 8 2023
++ 3dbucket: AFNI version=AFNI_23.1.05 (May 8 2023) [64-bit]
cp: No match.
---- done
++ have do_clean, cleaning up...
tee out.vlines.pb00.tcat.txt
find_variance_lines.tcsh -polort 22 -nerode 2 -rdir vlines.pb00.tcat pb00.sub-032163.r01.tcat+orig.HEAD
++ have nslices : 36
++ copying pb00.sub-032163.r01.tcat+orig.HEAD[0..$] to ts.0.orig.r01.nii.gz
++ 3dTcat: AFNI version=AFNI_23.1.05 (May 8 2023) [64-bit]
++ elapsed time = 52.3 s
++ creating automask from first dset
++ 3dAutomask: AFNI version=AFNI_23.1.05 (May 8 2023) [64-bit]
++ Authored by: Emperor Zhark
++ Loading dataset ts.0.orig.r01.nii.gz
++ Forming automask
+ Fixed clip level = 753.889526
+ Used gradual clip level = 573.789185 .. 992.627991
+ Number voxels above clip level = 18595
+ Clustering voxels ...
+ Largest cluster has 18333 voxels
+ Clustering voxels ...
+ Largest cluster has 17393 voxels
+ Filled 813 voxels in small holes; now have 18206 voxels
+ Filled 34 voxels in large holes; now have 18240 voxels
+ Clustering voxels ...
+ Largest cluster has 18236 voxels
+ Clustering non-brain voxels ...
+ Clustering voxels ...
+ Largest cluster has 129220 voxels
+ Mask now has 18236 voxels
++ 18236 voxels in the mask [out of 147456: 12.37%]
++ first 10 x-planes are zero [from R]
++ last 9 x-planes are zero [from L]
++ first 13 y-planes are zero [from P]
++ last 11 y-planes are zero [from A]
++ first 8 z-planes are zero [from I]
++ last 4 z-planes are zero [from S]
++ Output dataset ./mask.nii.gz
++ CPU time = 0.000000 sec
-- will apply mask mask.nii.gz
++ eroding mask.nii.gz by 2 voxels
++ no -frac option: defaulting to -union
++ processing 1 input dataset(s), NN=2...
++ padding all datasets by 0 (for dilations)
++ frac 0 over 1 volumes gives min count 0
++ voxel limits: 0 clipped, 7176 survived, 140280 were zero
++ writing result mask.nii.gz...
++ Output dataset ./mask.nii.gz
++ requiring 5 voxels in mask columns
++ 3dLocalstat: AFNI version=AFNI_23.1.05 (May 8 2023) [64-bit]
++ Authored by: Emperor Zhark
++ Number of voxels in mask = 7176
++ Neighborhood comprises 73 voxels
++ Start sub-brick [0]
++ Output dataset ./tmp.mask.col.count.nii.gz
++ 3dcalc: AFNI version=AFNI_23.1.05 (May 8 2023) [64-bit]
++ Authored by: A cast of thousands
++ Output dataset ./mask.nii.gz
-- detrend -polort 22, new eset = ts.1.det.r01.nii.gz
++ 3dTproject: AFNI version=AFNI_23.1.05 (May 8 2023) [64-bit]
++ Authored by: Cox the Algebraic (Linear)
** FATAL ERROR: -polort value can't be over 20 :-(
** Program compile date = May 8 2023
++ 3dTstat: AFNI version=AFNI_23.1.05 (May 8 2023) [64-bit]
++ Authored by: KR Hammett & RW Cox
** FATAL ERROR: Can't open dataset ts.1.det.r01.nii.gz
** Program compile date = May 8 2023
++ 3dcalc: AFNI version=AFNI_23.1.05 (May 8 2023) [64-bit]
++ Authored by: A cast of thousands
** FATAL ERROR: can't open dataset tmp.stdev.nii.gz
** Program compile date = May 8 2023
rm: cannot remove 'tmp.stdev.nii.gz': No such file or directory
** FATAL ERROR: Can't open dataset 'var.0.orig.r01.nii.gz'
** Program compile date = May 8 2023
echo ++ FINISHED AP: 20_ap_vox
++ FINISHED AP: 20_ap_vox
exit 0
Howdy-
OK, that is a long time series. The option for controlling the degree of polynomial in the regression model is this, which explains the default degree chosen (based on the run_length value, which is the total time of the EPIs), as well as how to adjust it and use the “-regress_bandpass …” option to do some baseline capturing, itself:
-regress_polort DEGREE : specify the polynomial degree of baseline
e.g. -regress_polort 2
default: 1 + floor(run_length / 150.0)
3dDeconvolve models the baseline for each run separately, using
Legendre polynomials (by default). This option specifies the
degree of polynomial. Note that this will create DEGREE * NRUNS
regressors.
The default is computed from the length of a run, in seconds, as
shown above. For example, if each run were 320 seconds, then the
default polort would be 3 (cubic).
* It is also possible to use a high-pass filter to model baseline
drift (using sinusoids). Since sinusoids do not model quadratic
drift well, one could consider using both, as in:
-regress_polort 2 \
-regress_bandpass 0.01 1
Here, the sinusoids allow every frequency from 0.01 on up to pass
(assuming the Nyquist frequency is <= 1), modeling the lower
frequencies as regressors of no interest, along with 3 terms for
polort 2.
So, for your resting state analysis, you could put in these options to your AP command:
-regress_polort 2 \
-regress_bandpass 0.01 1 \
as a reasonable starting point.
–pt
The only small point I might add is that adding “-radial_correlate_opts -polort 18” should have affected the @radial_correlate script as you hoped. There would be 2 -polort options in the command in the proc script, but yours would be later and therefore override the first (a subtle point).
But still, use the options Paul suggests.
A minor effect of this goes the opposite way though (with @radial_correlate). If you specify “-regress_polort 2”, that should change the one used for @radial_correlate to be just 2, which is probably lower than preferable. It does not have a bandpassing option.
So even with Paul’s suggestion, you might also add something like “-radial_correlate_opts -polort 9”, just to have more reasonable detrending for those QC steps.
- rick
Thank you all for your patience and valuable time. I took all your suggestions, and now the script is working normally.
Best,
Jewel