Problems running afni_proc/3dDeconvolve - hangs at polort matrix

Dear AFNI-community,

I’m having trouble getting 3dDeconvolve to run. It seems to hang after the “polort-only matrix condition” (see output below). The 3dDeconvolve command was created using afni_proc.py. Here is what that command looks like:

afni_proc.py -verb 2 -subj_id ${lc}${session} -dsets ./rundata/${lc}${session}_run*ss+orig.HEAD
-blocks tcat despike volreg blur mask scale regress
-test_stim_files yes
-volreg_interp -quintic
-volreg_zpad 4
-blur_size 3
-mask_dilate 3
-mask_apply epi
-regress_censor_motion 0.3
-regress_censor_outliers 0.15
-regress_bandpass 0.01 0.1
-regress_apply_mask
-regress_apply_mot_types demean deriv
-regress_stim_labels faces places objects scrambled
-regress_stim_files ./stimfiles/${monkey_uc}
${session}stim1_Faces.txt
./stimfiles/${monkey_uc}
${session}stim2_Places.txt
./stimfiles/${monkey_uc}
${session}stim3_Objects.txt
./stimfiles/${monkey_uc}
${session}_stim4_Scrambled.txt
-regress_opts_3dD
-jobs 2
-num_glt 4
-gltsym ‘SYM: +faces +places +objects +scrambled’ -glt_label 1 all_conds
-gltsym ‘SYM: +faces -places’ -glt_label 2 faces-places
-gltsym ‘SYM: +faces -objects’ -glt_label 3 faces-objects
-gltsym ‘SYM: +faces -scrambled’ -glt_label 4 faces-scrambled
-progress 500
-regress_polort 2

It represents a single session of 1200 TRs.
I am using the most up-to-date AFNI version, on a macbook pro (late 2013) with 16GB of RAM
“afni -ver => Precompiled binary macosx_10.7_Intel_64: Jul 22 2016 (Version AFNI_16.2.05)”

I have tried the following:

  1. using different platforms (different computers, linux version) - no luck. same error
  2. running only the most basic options with 3dDeconvolve
  3. I have tried 3dZcutup/3dZcat - and it worked once; but hasn’t worked since. Same error.
  4. using fewer TRs / analysing different sessions
  5. using fewer/more “-jobs”

Below is what I get (this is for running non-3dZcutup’d data. Obviously the memory requirements for the sliced data are much less)

Any thoughts? Could this be a memory issue? Are there perhaps too many (baseline) parameters?

Many thanks for any advice/tips/shared experiences!

Andrew

[i]++ Number of time points: 1200 (before censor) ; 1194 (after)

  • Number of parameters: 784 [780 baseline ; 4 signal]
    ++ total shared memory needed = 2,295,718,400 bytes (about 2.3 billion [giga])
    ++ mmap() memory allocated: 2,295,718,400 bytes (about 2.3 billion [giga])
    ++ Memory required for output bricks = 2,295,718,400 bytes (about 2.3 billion [giga])
    ++ Wrote matrix image to file X.jpg
    ++ Wrote matrix values to file X.xmat.1D
    ++ ========= Things you can do with the matrix file =========
    ++ (a) Linear regression with ARMA(1,1) modeling of serial correlation:

3dREMLfit -matrix X.xmat.1D
-input “pb04.ranger_MI00937.r01.scale+orig.HEAD pb04.ranger_MI00937.r02.scale+orig.HEAD pb04.ranger_MI00937.r03.scale+orig.HEAD pb04.ranger_MI00937.r04.scale+orig.HEAD pb04.ranger_MI00937.r05.scale+orig.HEAD pb04.ranger_MI00937.r06.scale+orig.HEAD pb04.ranger_MI00937.r07.scale+orig.HEAD pb04.ranger_MI00937.r08.scale+orig.HEAD pb04.ranger_MI00937.r09.scale+orig.HEAD pb04.ranger_MI00937.r10.scale+orig.HEAD pb04.ranger_MI00937.r11.scale+orig.HEAD pb04.ranger_MI00937.r12.scale+orig.HEAD”
-mask full_mask.ranger_MI00937+orig -fout -tout
-Rbuck stats.ranger_MI00937_REML -Rvar stats.ranger_MI00937_REMLvar
-Rfitts fitts.ranger_MI00937_REML -Rerrts errts.ranger_MI00937_REML -verb

++ N.B.: 3dREMLfit command above written to file stats.REML_cmd
++ (b) Visualization/analysis of the matrix via ExamineXmat.R
++ (c) Synthesis of sub-model datasets using 3dSynthesize
++ ==========================================================
++ Wrote matrix values to file X.nocensor.xmat.1D
++ ----- Signal+Baseline matrix condition [X] (1194x784): 3.64199 ++ VERY GOOD ++
++ ----- Signal-only matrix condition [X] (1194x4): 1 ++ VERY GOOD ++
++ ----- Baseline-only matrix condition [X] (1194x780): 3.54591 ++ VERY GOOD ++
++ ----- stim_base-only matrix condition [X] (1194x744): 3.26556 ++ VERY GOOD ++
++ ----- polort-only matrix condition [X] (1194x36): 1.02839 ++ VERY GOOD ++:[/i]

Hi all,

Just a followup:

If I remove the following from the 3dDeconvolve command, the analysis proceeds (on sliced data) MUCH MUCH quicker.

-ortvec ./${subj}.results/bandpass_rall.1D bandpass

Could this indicate that I do have too many baseline regressors? Does anyone have any experience with this/suggestions on how to include bandpassing without encountering this issue?

Many thanks!

A

Incidentally, I did run this on my system, and
it was not so slow getting started. The processing
IS very slow (because of all the sub-models that are
solved), but mine got past your hang point reasonably
quickly (maybe a minute? I wasn’t watching so
closely…).

  • rick