Execute 3dREMLfit in multithreaded mode from python

AFNI Message Board
1

Dear afni experts,

I would like to execute afni commands from python, e.g., using subprocess.check_call() or ipython’s ! notation.
However, it seems that commands with OpenMP multithread support, e.g., 3dREMLfit or 3dDeconvolve, only deploy a single thread when executed from python, dramatically slowing down the computation.

For example, when I run
subprocess.check_call([‘tcsh’, ‘stats.REML_cmd’])
or
!tcsh stats.REML_cmd
in ipython, it spawns only one thread and computes very slowly. But when I run the same script directly from the shell
$ tcsh stats.REML_cmd
it spawns 15 thread and runs considerably faster.

Is there a way to execute 3dREMLfit in multithreaded mode from python?

Thanks for help~

2

Both of your commands (subprocess from python
and ‘!’ from ipython) use multi-threaded OpenMP
on my system.

What is the output of “afni -ver”? What OS and
OS version are you running?

  • rick
3

Good news:)

Here is my system info that might be relevant

% afni -ver
Precompiled binary macosx_10.7_Intel_64: Nov 23 2016 (Version AFNI_16.3.13)

% system_profiler SPSoftwareDataType
System Version: macOS 10.12 (16A323)
Kernel Version: Darwin 16.0.0
Boot Volume: C
Boot Mode: Normal
Secure Virtual Memory: Enabled
System Integrity Protection: Disabled
Time since boot: 6 days 8:11

% ipython
Python 3.5.2 |Anaconda custom (x86_64)| (default, Jul 2 2016, 17:52:12)

% cat .bashrc

For afni

export PATH=$PATH:/Users/herrlich10/abin
export DYLD_LIBRARY_PATH=/opt/X11/lib/flat_namespace
export PYTHONPATH=/usr/local/lib/python2.7/site-packages

For freesurfer

export FREESURFER_HOME=/Applications/freesurfer
source $FREESURFER_HOME/SetUpFreeSurfer.sh

Do you see any suspicious items?

4

Sorry, I’ve overlooked an important detail:

Previously, I tested both subprocess and ! notation in jupyter notebook, and got the single threaded result.
I’ve just re-tested them in vanilla ipython console, and as rick said, both of them worked as expected with OpenMP enabled.

What’s wrong with jupyter notebook, then?

5

Reinstalled anaconda (Python 3.5.2 |Anaconda 4.2.0 (x86_64)), now OpenMP works in jupyter notebook.