I keep getting a message afni_proc.py: 1: Cannot fork and can't run my analysis.

[hoptman]

AFNI version info (afni -ver):

code text  # or delete if not needed

Hi all,

I have truncated the time series and resampled them to 3.5mm3, and I've also set OMP_NUM_THREADS=1, but it's not helping. I'm using
Precompiled binary linux_ubuntu_24_64: Feb 12 2025 (Version AFNI_25.0.06 'Severus Alexander'), and I have 64GB of memory with an 8.1GB swap space.

Thanks,

Matt

I'm running:

afni_proc.py \

        -subj_id sub-3028_blip  \\

        -blocks          align tlrc volreg mask combine \\

                         scale regress                                   \\

        -copy_anat 	     anat/sub-3028_ses-MRI1_T1w.nii.gz		\\

        -dsets_me_echo   func/epi_small_1_3mm.nii.gz'\[0..100\]' \\

        -dsets_me_echo   func/epi_small_2_3mm.nii.gz'\[0..100\]' \\

        -dsets_me_echo   func/epi_small_3_3mm.nii.gz'\[0..100\]' \\

	    -dsets_me_echo   func/epi_small_1_3mm.nii.gz'\[0.100\]' \\

        -echo_times      12.8 32.87 52.94 73.01                     \\

        -blip_forward_dset 'fmap/sub-3028_ses-MRI1_task-rest_dir-AP-part-phase_epi.nii.gz\[0\]' \\

        -blip_reverse_dset 'fmap/sub-3028_ses-mRI1_task-rest_dir-PA-part-phase_epi.nii.gz\[0\]' \\

        -mask_epi_anat   yes                                        \\

        -align_opts_aea -cost lpc+ZZ -giant_move -check_flip \\

        -volreg_align_to MIN_OUTLIER \\

        -volreg_align_e2a           \\

        -volreg_tlrc_warp \\

        -tlrc_base MNI152_2009_template_SSW.nii.gz \\

        -regress_apply_mot_types demean deriv \\

        -regress_motion_per_run \\

        -regress_censor_motion 0.5 \\

        -regress_censor_outliers 0.05 \\

        -html_review_style pythonic \\

        -combine_method  OC

[rickr]

Hi Matt,

Is that happening during the regression? We don't have too many case where a process is forked, though it could even be afni.

Would you please post the exact error message, along with many lines leading up to it?

Thanks,

-rick

[hoptman]

Hi Rick,

I never get to that point. I submit the afni_proc.py and get that message after a short delay. The regression is supposed to detrend, regress motion (demeaned and derivatives), censor motion and outliers, and save the errts.

Of note, this script worked with full image series and native image resolutiion.

afni_proc.py
-blocks align tlrc volreg mask combine blur
scale regress
-copy_anat anat/subj_T1w.nii.gz
-dsets_me_echo func/subj_task-rest_echo-1_part-mag_bold.nii.gz
-dsets_me_echo func/subj_task-rest_echo-2_part-mag_bold.nii.gz
-dsets_me_echo func/subj_task-rest_echo-3_part-mag_bold.nii.gz
-dsets_me_echo func/subj_task-rest_echo-1_part-mag_bold.nii.gz
-echo_times 12.8 32.87 52.94 73.01
-mask_epi_anat yes
-blur_size 4
-align_opts_aea -cost lpc+ZZ -giant_move -check_flip
-volreg_align_to MIN_OUTLIER
-volreg_align_e2a
-volreg_tlrc_warp
-tlrc_base MNI152_2009_template_SSW.nii.gz
-regress_apply_mot_types demean deriv
-regress_motion_per_run
-regress_censor_motion 0.5
-regress_censor_outliers 0.05
-regress_est_blur_epits
-tlrc_NL_warp
-blur_in_mask yes
-html_review_style pythonic
-combine_method OC

[ptaylor]

Hi, Matt-

I took your command and vertically aligned opts, for easier reading:

afni_proc.py                                                                 \
    -subj_id                  sub-3028_blip                                  \
    -blocks                   align tlrc volreg mask combine scale regress   \
    -copy_anat                anat/sub-3028_ses-MRI1_T1w.nii.gz              \
    -dsets_me_echo            func/epi_small_1_3mm.nii.gz'\[0..100\]'        \
    -dsets_me_echo            func/epi_small_2_3mm.nii.gz'\[0..100\]'        \
    -dsets_me_echo            func/epi_small_3_3mm.nii.gz'\[0..100\]'        \
    -dsets_me_echo            func/epi_small_1_3mm.nii.gz'\[0.100\]'         \
    -echo_times               12.8 32.87 52.94 73.01                         \
    -blip_forward_dset        'fmap/sub-3028_ses-MRI1_task-rest_dir-AP-part-phase_epi.nii.gz\[0\]' \
    -blip_reverse_dset        'fmap/sub-3028_ses-mRI1_task-rest_dir-PA-part-phase_epi.nii.gz\[0\]' \
    -mask_epi_anat            yes                                            \
    -align_opts_aea           -cost lpc+ZZ                                   \
                              -giant_move                                    \
                              -check_flip                                    \
    -volreg_align_to          MIN_OUTLIER                                    \
    -volreg_align_e2a                                                        \
    -volreg_tlrc_warp                                                        \
    -tlrc_base                MNI152_2009_template_SSW.nii.gz                \
    -regress_apply_mot_types  demean deriv                                   \
    -regress_motion_per_run                                                  \
    -regress_censor_motion    0.5                                            \
    -regress_censor_outliers  0.05                                           \
    -html_review_style        pythonic                                       \
    -combine_method           OC

One issue is that I don't think you need both the backslashes before the square-bracket selectors and the quotes (esp. single quotes, which might be a particular problem here) in Lines 10-11.

Also, in line 8, there is only one dot . in the subbrick selector to denote range, where there should be two .. (as in the other cases just above)

I am not sure if these are causing the other issues, but could you try this:

afni_proc.py                                                                 \
    -subj_id                  sub-3028_blip                                  \
    -blocks                   align tlrc volreg mask combine scale regress   \
    -copy_anat                anat/sub-3028_ses-MRI1_T1w.nii.gz              \
    -dsets_me_echo            func/epi_small_1_3mm.nii.gz"[0..100]"          \
    -dsets_me_echo            func/epi_small_2_3mm.nii.gz"[0..100]"          \
    -dsets_me_echo            func/epi_small_3_3mm.nii.gz"[0..100]"          \
    -dsets_me_echo            func/epi_small_1_3mm.nii.gz"[0..100]"          \
    -echo_times               12.8 32.87 52.94 73.01                         \
    -blip_forward_dset        "fmap/sub-3028_ses-MRI1_task-rest_dir-AP-part-phase_epi.nii.gz[0]" \
    -blip_reverse_dset        "fmap/sub-3028_ses-mRI1_task-rest_dir-PA-part-phase_epi.nii.gz[0]" \
    -mask_epi_anat            yes                                            \
    -align_opts_aea           -cost lpc+ZZ                                   \
                              -giant_move                                    \
                              -check_flip                                    \
    -volreg_align_to          MIN_OUTLIER                                    \
    -volreg_align_e2a                                                        \
    -volreg_tlrc_warp                                                        \
    -tlrc_base                MNI152_2009_template_SSW.nii.gz                \
    -regress_apply_mot_types  demean deriv                                   \
    -regress_motion_per_run                                                  \
    -regress_censor_motion    0.5                                            \
    -regress_censor_outliers  0.05                                           \
    -html_review_style        pythonic                                       \
    -combine_method           OC

--pt

ps: For reference, your other command, formatted similarly (but which contains no subbrick selection), is:

afni_proc.py                                                                 \
    -blocks                   align tlrc volreg mask combine blur scale      \
                              regress                                        \
    -copy_anat                anat/subj_T1w.nii.gz                           \
    -dsets_me_echo            func/subj_task-rest_echo-1_part-mag_bold.nii.gz \
    -dsets_me_echo            func/subj_task-rest_echo-2_part-mag_bold.nii.gz \
    -dsets_me_echo            func/subj_task-rest_echo-3_part-mag_bold.nii.gz \
    -dsets_me_echo            func/subj_task-rest_echo-1_part-mag_bold.nii.gz \
    -echo_times               12.8 32.87 52.94 73.01                         \
    -mask_epi_anat            yes                                            \
    -blur_size                4                                              \
    -align_opts_aea           -cost lpc+ZZ                                   \
                              -giant_move                                    \
                              -check_flip                                    \
    -volreg_align_to          MIN_OUTLIER                                    \
    -volreg_align_e2a                                                        \
    -volreg_tlrc_warp                                                        \
    -tlrc_base                MNI152_2009_template_SSW.nii.gz                \
    -regress_apply_mot_types  demean deriv                                   \
    -regress_motion_per_run                                                  \
    -regress_censor_motion    0.5                                            \
    -regress_censor_outliers  0.05                                           \
    -regress_est_blur_epits                                                  \
    -tlrc_NL_warp                                                            \
    -blur_in_mask             yes                                            \
    -html_review_style        pythonic                                       \
    -combine_method           OC

[rickr]

And concatenating the 2 commands into 1 around -compare_opts_vs_opts shows that the basic differences are blip datasets in one and blurring in the other.

-rick

[ptaylor]

... and just for those keeping score at home, @rickr 's excellent suggestion was to run this command to compare the sets of opts, by providing 2 sets of AP options separated by -compare_opts_vs_opts (see that opt hanging out in line 26 here):

afni_proc.py                                                                 \
    -subj_id                  sub-3028_blip                                  \
    -blocks                   align tlrc volreg mask combine scale regress   \
    -copy_anat                anat/sub-3028_ses-MRI1_T1w.nii.gz              \
    -dsets_me_echo            func/epi_small_1_3mm.nii.gz"[0..100]"          \
    -dsets_me_echo            func/epi_small_2_3mm.nii.gz"[0..100]"          \
    -dsets_me_echo            func/epi_small_3_3mm.nii.gz"[0..100]"          \
    -dsets_me_echo            func/epi_small_1_3mm.nii.gz"[0..100]"          \
    -echo_times               12.8 32.87 52.94 73.01                         \
    -blip_forward_dset        "fmap/sub-3028_ses-MRI1_task-rest_dir-AP-part-phase_epi.nii.gz[0]" \
    -blip_reverse_dset        "fmap/sub-3028_ses-mRI1_task-rest_dir-PA-part-phase_epi.nii.gz[0]" \
    -mask_epi_anat            yes                                            \
    -align_opts_aea           -cost lpc+ZZ                                   \
                              -giant_move                                    \
                              -check_flip                                    \
    -volreg_align_to          MIN_OUTLIER                                    \
    -volreg_align_e2a                                                        \
    -volreg_tlrc_warp                                                        \
    -tlrc_base                MNI152_2009_template_SSW.nii.gz                \
    -regress_apply_mot_types  demean deriv                                   \
    -regress_motion_per_run                                                  \
    -regress_censor_motion    0.5                                            \
    -regress_censor_outliers  0.05                                           \
    -html_review_style        pythonic                                       \
    -combine_method           OC                                             \
  -compare_opts_vs_opts                                            \
    -blocks                   align tlrc volreg mask combine blur scale      \
                              regress                                        \
    -copy_anat                anat/subj_T1w.nii.gz                           \
    -dsets_me_echo            func/subj_task-rest_echo-1_part-mag_bold.nii.gz \
    -dsets_me_echo            func/subj_task-rest_echo-2_part-mag_bold.nii.gz \
    -dsets_me_echo            func/subj_task-rest_echo-3_part-mag_bold.nii.gz \
    -dsets_me_echo            func/subj_task-rest_echo-1_part-mag_bold.nii.gz \
    -echo_times               12.8 32.87 52.94 73.01                         \
    -mask_epi_anat            yes                                            \
    -blur_size                4                                              \
    -align_opts_aea           -cost lpc+ZZ                                   \
                              -giant_move                                    \
                              -check_flip                                    \
    -volreg_align_to          MIN_OUTLIER                                    \
    -volreg_align_e2a                                                        \
    -volreg_tlrc_warp                                                        \
    -tlrc_base                MNI152_2009_template_SSW.nii.gz                \
    -regress_apply_mot_types  demean deriv                                   \
    -regress_motion_per_run                                                  \
    -regress_censor_motion    0.5                                            \
    -regress_censor_outliers  0.05                                           \
    -regress_est_blur_epits                                                  \
    -tlrc_NL_warp                                                            \
    -blur_in_mask             yes                                            \
    -html_review_style        pythonic                                       \
    -combine_method           OC

The end result is a comparison/review table, so you can spot any differences that might matter, such as in the selected blocks, or options, or arguments provided to the options. In this case, that table looks like this---so, the first set of options ("current") is missing the blur bock, relative to the second set ("target"), etc.:

===========================================================================
==== comparing (current) 'command_1' vs (target) 'command_2'
     (for more detail, consider "-verb 2")

==========  missing option(s) : 4
   -blur_size                4
   -regress_est_blur_epits   
   -tlrc_NL_warp             
   -blur_in_mask             yes

==========  extra option(s) : 3
   -subj_id                  sub-3028_blip
   -blip_forward_dset        fmap/sub-3028_ses-MRI1_task-rest_dir-AP-part- ...
   -blip_reverse_dset        fmap/sub-3028_ses-mRI1_task-rest_dir-PA-part- ...

==========  differing option(s) : 6
   -blocks                 
      current  : align tlrc volreg mask combine scale regress
      target   : align tlrc volreg mask combine blur scale reg ...

   -copy_anat                (verb=1; skipping details)

   -dsets_me_echo            (verb=1; skipping details)

==========  fewer applied option(s) : 0

==========  more applied option(s) : 0

--pt

[hoptman]

Thank you both. I don't know why my text, which is reasonably aligned in my terminal window gets so messed up. Thanks for catching the typo. I didn't put in the backslashes. It might be a linux to windows issue in copy the text. I'll try these solutions, and see what happens, though the first set of commands is still giving me the "forking" problem. On to the second set of commands, though it looks like you already did that. I am running the afni_proc.py command as: "tcsh -xef afni_proc.py". Is that ok?

[ptaylor]

Hi, Matthew-

If you still get an error message, can you please copy+paste the full text of it. Having that level of specificity really helps in troubleshooting (in addition to having the exact command, which you have already provided).

The afni_proc.py command can be run in any shell: tcsh, bash, etc. Dealing with squarebrackets and how they are escaped might differ across shells, depending how it is done, but the above versions of the commands that we have copied and pasted should be save in any shell. So, you can put them in a bash or tcsh or zsh script and execute it.

After the AP command runs and creates the proc script, it will offer you a couple suggestions of running that script to do the processing, with convenient logging of terminal text. At that point, you need to know what shell you are using, because the syntax of sending the terminal text to be logged is shell-specific.

To know what shell you are running in, you can type:

echo $0

... and that will help you decide which command to copy+paste.

thanks,
pt

[hoptman]

Hi Paul,

I'm using bash, which I was pretty sure was what I was using. Strangely (see below), the double quotes are not showing in my terminal window.

Here is the the result of the "tcsh -xef afni_proc.py" command:

afni_proc.py -subj_id sub-3028_blip -blocks align tlrc volreg mask combine scale regress -copy_anat anat/sub-3028_ses-MRI1_T1w.nii.gz -dsets_me_echo func/epi_small_1_3mm.nii.gz[0..100] -dsets_me_echo func/epi_small_2_3mm.nii.gz[0..100] -dsets_me_echo func/epi_small_3_3mm.nii.gz[0..100] -dsets_me_echo func/epi_small_1_3mm.nii.gz[0..100] -echo_times 12.8 32.87 52.94 73.01 -blip_forward_dset fmap/sub-3028_ses-MRI1_task-rest_dir-AP-part-phase_epi.nii.gz[0] -blip_reverse_dset fmap/sub-3028_ses-mRI1_task-rest_dir-PA-part-phase_epi.nii.gz[0] -mask_epi_anat yes -align_opts_aea -cost lpc+ZZ -giant_move -check_flip -volreg_align_to MIN_OUTLIER -volreg_align_e2a -volreg_tlrc_warp -tlrc_base MNI152_2009_template_SSW.nii.gz -regress_apply_mot_types demean deriv -regress_motion_per_run -regress_censor_motion 0.5 -regress_censor_outliers 0.05 -html_review_style pythonic -combine_method OC
./afni_proc.py: 1: Cannot fork

As you can see, the error message isn't that informative, and I never get a proc script out of the command

[ptaylor]

Hi, Matthew-

I think when the command is echoed, it will be expected that the quotes don't get displayed.

And wow, I have not run into that output error before. From searching online, it seems that is something at the system level, about not being able to start up new processes, because the max number of allowed running processes has been reached on the OS. Is that a shared computer or are lots of things running? (It also seems a bit odd that it displays afni_proc.py with a preceding ./.)

Out of curiosity, can you run any command at the terminal, such as:

ls

or (PT: now edited, to fix wrongly copied filename):

3dcopy func/epi_small_1_3mm.nii.gz  tmp.nii.gz

(and you can remove the tmp.nii.gz that is created in the second case, if it does get generated)?

--pt

[hoptman]

Hi Paul,

ls works, as does pretty much every other command. The 3dcopy command gives me:

++ 3dcopy: AFNI version=AFNI_25.0.06 (Feb 12 2025) [64-bit]
** FATAL ERROR: 3dcopy only takes one input AND one output filename after options.
Have at least one too many parameters:
func/epi_small_1_3mm.nii.gz[0..100]
-dsets_me_echo
func/epi_small_2_3mm.nii.gz
** Program compile date = Feb 12 2025

I had followed some guidance from Microsoft Edge's AI and had truncated and resampled the time series from 305 volumes and 2.5mm3 resolution to 100 volumes and 3mm3 resolution. I am WAY below my ulimit in terms of running processes.

[ptaylor]

Sorry, that was a copy+paste error by me; I had meant:

3dcopy func/epi_small_1_3mm.nii.gz  tmp.nii.gz

[ptaylor]

And just to check, when you write that you ran "tcsh -xef afni_proc.py", what does that mean?

I would run it by putting the full afni_proc.py command into a text file, say a tcsh script "do_ap.tcsh". Then, you could execute that script from the command line with:

tcsh -xef do_ap.tcsh

Is that what you mean? You cannot run tcsh ... directly on afni_proc.py itself (it is a Python file).

--pt

[rickr]

Try adding -verb 3 to the command, and see if that gives an idea where the failed forking is happening. The program does use subprocess for running commands, but that should not be specific to blip.

Given the output, it seems like it might be failing before afni_proc.py even begins. Are the shells and environments identical between the 2 cases?

-rick

[hoptman]

OK, that worked.

[hoptman]

Hi Paul and Rick,

I'm sorry, I actually was running "tcsh -xef afni_proc.py". My gut is it caused some kind of recursion error, which is why it took a relatively long time to fail.

Again, my apologies. Now everything is working.

Best wishes,

Matt

[ptaylor]

Hi, Matt-

OK, great, glad that sorted it. As a bonus, I think asking the question here probably also added some benefit for troubleshooting the selector brackets (not needing backslashes) and the dots.

So, hopefully the processing runs smoothly from here.

--pt