AFNI_proc Preprocessing does not finish

Philipp · October 17, 2021, 10:27am

Dear all,

I ran upon a strange problem. Let me provide you with an overview first:

A resting-state analysis (preprocessing) was done for a resting-state run (1 run per subject; 25 subjects overall). This session also included running SSwarper.
Now I would like to apply the same script for a task-based session (6 runs per subject instead of 1 run as above). Note: why do I apply a “resting-state” script on a task-based run? The reason simply is that I am interested in the brain’s dynamics, such as Power Law Exponent (PLE), and I would like to compare the real resting-state run with the task-runs (processed as if they were resting-state runs). This then allows me to properly compare variables like the PLE between both sessions.

To save time and as you know, it is no longer required to run SSwarper again for the second sessions’ preprocessing, as SSwarper was already done before. This is my script for the task-based runs (processed as “resting-state”, i.e., without the GLM).

TaskDynamics Preprocessing

for subject in Subject1 Subject2 Subject3 Subject4 Subject5 Subject6 Subject7 Subject8 Subject9 Subject10 Subject11 Subject12 Subject13 Subject14 Subject15 Subject16 Subject17 Subject18 Subject19 Subject20 Subject21 Subject22 Subject23 Subject24 Subject25
do

Set directories

directory_subjects=/volumes/sandisk/fmri/dataset/subjects/$subject
directory_subjects_general=/volumes/sandisk/fmri/dataset/subjects
directory_fMRIruns=$directory_subjects/raw
directory_anatomical=$directory_subjects/raw
directory_SSwarper=$directory_subjects/SSwarper

cd $directory_subjects

AFNI_proc.py

afni_proc.py
-subj_id ${subject}_TaskDynamics
-out_dir $directory_subjects/Preprocessing_TaskDynamics
-dsets
$directory_fMRIruns/run*to3d+orig.HEAD
-blocks despike tshift align tlrc volreg mask blur regress
-copy_anat $directory_SSwarper/anatSS.$subject.nii
-anat_has_skull no
-tcat_remove_first_trs 4
-align_opts_aea -cost nmi
-volreg_align_e2a
-volreg_align_to MIN_OUTLIER
-volreg_tlrc_warp -tlrc_base MNI152_2009_template_SSW.nii.gz
-tlrc_NL_warp
-tlrc_NL_warped_dsets
$directory_subjects/SSwarper/anatQQ.$subject.nii
$directory_subjects/SSwarper/anatQQ.$subject.aff12.1D
$directory_subjects/SSwarper/anatQQ.${subject}_WARP.nii
-volreg_post_vr_allin yes
-volreg_pvra_base_index MIN_OUTLIER
-mask_segment_anat yes
-mask_segment_erode yes
-regress_anaticor
-regress_ROI WMe
-regress_apply_mot_types demean deriv
-regress_motion_per_run
-regress_censor_motion 0.2
-regress_censor_outliers 0.05
-regress_bandpass 0.02 0.2
-blur_size 6.0
-regress_est_blur_epits
-regress_est_blur_errts
-html_review_style pythonic
-execute

done

What is the problem? The problem is that even after 8-12 hours of running AFNI_proc it still computes the first subject (I already tried it twice). All applications beside the terminal are closed (2013 MacBook, 2.4 GHz dual core i5, 4 GB Ram). I know that this laptop is very slow compared to today’s standards. But ±10 hours per subject and it is still preprocessing? Running AFNI_proc for 1 subject took around an hour for the resting-state run.

Is there a mistake in my preprocessing script? I would be very happy if you could share your ideas.

Philipp

rickr · October 18, 2021, 2:28pm

Hi Philipp,

It would be good to make sure at what stage it takes so much time, but the regression step is the most likely. Please see this thread or post with some comments on thrashing. Try out the -regress_compute_fitts option, which saves a fair amount of RAM during the regression step.

The bandpass filtering tends to add many regressors, which in turn slows the regression down too, though at least your pass band range is larger.

rick

Philipp · October 18, 2021, 3:30pm

Hello Rick,

thanks for the tip with -regress_compute_fitts. I will use it for the genuine task-based session. One question: -regress_compute_fitts should not change anything concerning the time needed by AFNI_proc in my script shown in the first post, correct? Thats because no fitts file is created here anyway, as the GLM is missing and I preprocess the task-based runs “as if” they were resting-state runs.

Am I correct? Or does -regress_compute_fitts still speed up the preprocessing even though in the eyes of AFNI this (see my first post) is a “resting-state” run? That would not really make sense to me.

Philipp

rickr · October 18, 2021, 5:13pm

Hi Philipp,

Oh yes, you are right about that. But can you clarify exactly which command is taking so long?

The details about thrashing and swap space still apply though. If the process is taking a very long time, that is likely what is happening. But to have a better idea of what to suggest, it would be helpful to know what step is lagging. Can you tell?

Thanks,

rick

Philipp · October 20, 2021, 6:44am

Dear Rick,

I started AFNI_proc again last evening to let it run overnight. Now, around 14 hours later, it is still processing subject1 (as you can see on the screenshot below). This screenshot also shows you where AFNI seems to be stuck.

Additional information:

AFNI_proc took around 1 hour for the resting-state run per subject. Number of time steps (resting-state): 360.
Now lets compare this to the task-based runs: Number of time steps: 1180. Again, there are also 6 runs here instead of 1.

Does the processing time scale in a linear fashion? Even if that would be the case, that is, around 3 hours per run (from 360 to 1180) times 6 makes 18 hours per subject. Or would you assume that something is going wrong here?

Philipp

Philipp · October 20, 2021, 7:59am

Ok, Rick, an update. AFNI just continued with the preprocessing (it provides me with new output in the terminal), but it is still working on subject1.

I am sorry if this thread might be silly from me, because the reason probably is that my laptop is just extremly slow. I already feel a bit bad for this. But can this really be? I mean that it is THAT slow for the 6 task-based runs compared to the 1 resting-state run?