Multiple Jobs / Parallel processing - 3dQwarp

AFNI Message Board
1

Hello,
I am running preprocessing pipelines in hundreds of subjects that come from open source initiatives.

A few months ago I was doing that on my MacOS Sierra laptop.
(MacBook Pro (13-inch, 2017, Two Thunderbolt 3 ports, 2.3 GHz Intel Core i5, RAM: 8 GB 2133 MHz LPDDR3 etc)

the fastest way to do that was to try and parallelise the procedure and I was running 15-20 subject on a terminal window ( shell-script pile up of the pipeline) and could have had 3 Terminal windows open and running, maximum with the specs of the laptop.

I am working on an Ubuntu 17.10 Artful now
RAM 32Gb
Intel Core i7- @ 4.20GHz x 8
OS type 64-bit

Which is evidently much more powerful than the laptop I was using.
In this desktop it is impossible to have more than 1 window open (its impossible to parallelise the pipeline), and this is much more evident when 3dQwarp is running.

When 3 windows (pipelines) are open looping over 20 subjects each 3dQwarp needs close to 30 hours to finish 1 subject
When 2 windows (pipelines) are open looping over 20 subjects each 3dQwarp needs close to 11 hours to finish 1
When 1 window is open with the same (or more it doesn’t matter) subjects it needs less than 30 minutes to finish 1 subject.

I am using the latest afni programms in both computers: Mar 6 2018 (Version AFNI_18.0.24)
However, AFNI on the Ubuntu desktop is downloaded from Neurodebian.

What do you think the issue might be?

IS there anything I can do to allow parallel processing of AFNI programs in this desktop? This was the norm in Mac but perhaps there is something extra that needs to be added in Ubuntu for this to happen, safely.

Thanks,

Stavros

2

Hi, Stavros-

3dQwarp is inherently parallelized using OpenMP. It can look for multiple threads to use while processing-- and using multiple threads is definitely recommended. You can set the maximum number of threads per job using an environment variable in your ~/.bashrc or ~/cshrc file it is called “OMP_NUM_THREADS”. To know your shell, type “echo $0”.

For example, you could set it to be 8 by putting this in ~/.bashrc:


# bash
export OMP_NUM_THREADS=8

or this in your ~/.cshrc:


# tcsh
setenv OMP_NUM_THREADS 8

and then source either file ( “source ~/.bashrc” or “source ~/.cshrc”, whichever shell you are using).

To see how many threads you are currently allowing OpenMP programs like 3dQwarp, type:


3dQwarp -hview

on the commandline, and the OpenMP thread number should be shown in the terminal (and the “-hview” opens the help for that program in a text editor).

–pt

3

That fixed it!
Thanks a lot.

Stavros