3dttest++ clustSim error

AFNI Message Board
1

Hello AFNI. I am running the following group comparison without an specific mask:
3dttest++
-prefix HighvsLowICG_noCov_noMask
-prefix_clustsim HighvsLowICG_noCov_noMask
-tempdir /scratch/global/lbohorquez/PCASL_analysis/results_noMask/temp
-center SAME
-Clustsim 24
-toz
-setA HighICG
-230 r_CBF_GMC_80.nii.gz
-…
-setB LowICG
-172 r_CBF_GMC_80.nii.gz
- …

It seems that the t-test runs but at the clustSim moment, something is failing, any suggestions of what am I doing wrong?

++ 3dttest++: AFNI version=AFNI_19.3.18 (Dec 27 2019) [64-bit]
++ Authored by: Zhark++
++ Number of -Clustsim threads set to 24
++ option -setA :: processing as LONG form (label label dset label dset …)
++ have 21 volumes corresponding to option ‘-setA’
++ option -setB :: processing as LONG form (label label dset label dset …)
++ have 18 volumes corresponding to option ‘-setB’
++ no mask ==> processing all 902629 voxels
++ 2-sample test: ‘-AminusB’ option is assumed
++ 2-sample test: results will be HighICG - LowICG
++ loading -setA datasets
^[[7m*+ WARNING:^[[0m 3dttest++ -setA :: 790863 vectors are constant
++ loading -setB datasets
^[[7m*+ WARNING:^[[0m 3dttest++ -setB :: 790863 vectors are constant
++ Memory usage now = 321,648,610 (322 million)
++ t-testing:0123456789.0123456789.0123456789.0123456789.0123456789.!

  • skipped 793799 voxels completely for having constant data
    ++ ---------- End of analyses – freeing workspaces ----------
    ++ Memory usage now = 166,374,694 (166 million)
    ++ Creating FDR curves in output dataset
    ^[[7m*+ WARNING:^[[0m Smallest FDR q [1 HighICG-LowICG_Zscr] = 0.164516 ==> few true single voxel detections
    ++ Smallest FDR q [3 HighICG_Zscr] = 1.02322e-14
    ++ Smallest FDR q [5 LowICG_Zscr] = 3.17844e-14
  • Added 3 FDR curves to dataset
    ++ Output dataset ./HighvsLowICG_noCov_noMask.resid.nii
  • 2-sample test: results are HighICG - LowICG
    ++ ================ Starting -Clustsim calculations ================
  • === temporary files will have prefix HighvsLowICG_noCov_noMask ===
  • === running 24 -randomsign jobs (417 iterations per job) ===
    ++ 3dttest++: AFNI version=AFNI_19.3.18 (Dec 27 2019) [64-bit]
    ++ Authored by: Zhark++
    ++ option -setA :: processing as SHORT form (all values are datasets)
    ++ have 21 volumes corresponding to option ‘-setA’
    ++ option -setB :: processing as SHORT form (all values are datasets)
    ++ have 18 volumes corresponding to option ‘-setB’
    ++ random seeds are 416376842 730536107
    ++ no mask ==> processing all 902629 voxels
    ++ 2-sample test: ‘-AminusB’ option is assumed
    ++ 2-sample test: results will be SetA - SetB
    ++ loading -setA datasets
    ^[[7m*+ WARNING:^[[0m 3dttest++ -setA :: 793799 vectors are constant
    ++ loading -setB datasets
    ^[[7m*+ WARNING:^[[0m 3dttest++ -setB :: 793799 vectors are constant
    ++ Memory usage now = 159,195,775 (159 million)
    ++ t-test randomsign/permute:0123456789.012sh: line 3: 87130 Killed 3dttest++ -DAFNI_AUTOMATIC_FDR=NO -DAFNI_DONT_LOGFILE=YES -randomsign 417 -nomeans -toz -seed 423043748 737203013 -no1sam -permute -setA HighvsLowICG_noCov_noMask.resid.nii’[0…20]’ -setB HighvsLowICG_noCov_noMask.resid.nii’[21…$]’ -prefix /scratch/global/lbohorquez/PCASL_analysis/results_noMask/temp/HighvsLowICG_noCov_noMask.0006.nii &>/dev/null
    sh: line 3: 87143 Killed 3dttest++ -DAFNI_AUTOMATIC_FDR=NO -DAFNI_DONT_LOGFILE=YES -randomsign 417 -nomeans -toz -seed 427488352 741647617 -no1sam -permute -setA HighvsLowICG_noCov_noMask.resid.nii’[0…20]’ -setB HighvsLowICG_noCov_noMask.resid.nii’[21…$]’ -prefix /scratch/global/lbohorquez/PCASL_analysis/results_noMask/temp/HighvsLowICG_noCov_noMask.0010.nii &>/dev/null
    .
    .
    .
  • 3dttest++ ===== simulation jobs have finished (7286.4 s elapsed)
  • 3dttest++ ===== starting 3dClustSim A: elapsed = 7291.2 s
    ++ 3dClustSim: AFNI version=AFNI_19.3.18 (Dec 27 2019) [64-bit]
    ++ Authored by: RW Cox and BD Ward
    ^[[7m** ERROR:^[[0m -inset ‘/scratch/global/lbohorquez/PCASL_analysis/results_noMask/temp/HighvsLowICG_noCov_noMask.0000.nii’: failure to open dataset
    ^[[7m** ERROR:^[[0m -inset ‘/scratch/global/lbohorquez/PCASL_analysis/results_noMask/temp/HighvsLowICG_noCov_noMask.0001.nii’: failure to open dataset
    ^[[7m** ERROR:^[[0m -inset ‘/scratch/global/lbohorquez/PCASL_analysis/results_noMask/temp/HighvsLowICG_noCov_noMask.0002.nii’: failure to open dataset
    ^[[7m** ERROR:^[[0m -inset ‘/scratch/global/lbohorquez/PCASL_analysis/results_noMask/temp/HighvsLowICG_noCov_noMask.0004.nii’: failure to open dataset
    ^[[7m** ERROR:^[[0m -inset ‘/scratch/global/lbohorquez/PCASL_analysis/results_noMask/temp/HighvsLowICG_noCov_noMask.0005.nii’: failure to open dataset
    ^[[7m** ERROR:^[[0m -inset ‘/scratch/global/lbohorquez/PCASL_analysis/results_noMask/temp/HighvsLowICG_noCov_noMask.0006.nii’: failure to open dataset
    ^[[7m** ERROR:^[[0m -inset ‘/scratch/global/lbohorquez/PCASL_analysis/results_noMask/temp/HighvsLowICG_noCov_noMask.0007.nii’: failure to open dataset
    ^[[7m** ERROR:^[[0m -inset ‘/scratch/global/lbohorquez/PCASL_analysis/results_noMask/temp/HighvsLowICG_noCov_noMask.0008.nii’: failure to open dataset
    ^[[7m** ERROR:^[[0m -inset ‘/scratch/global/lbohorquez/PCASL_analysis/results_noMask/temp/HighvsLowICG_noCov_noMask.0009.nii’: failure to open dataset
    ^[[7m** ERROR:^[[0m -inset ‘/scratch/global/lbohorquez/PCASL_analysis/results_noMask/temp/HighvsLowICG_noCov_noMask.0010.nii’: failure to open dataset
    ^[[7m** ERROR:^[[0m -inset ‘/scratch/global/lbohorquez/PCASL_analysis/results_noMask/temp/HighvsLowICG_noCov_noMask.0012.nii’: failure to open dataset
    ^[[7m** ERROR:^[[0m -inset ‘/scratch/global/lbohorquez/PCASL_analysis/results_noMask/temp/HighvsLowICG_noCov_noMask.0014.nii’: failure to open dataset
    ^[[7m** ERROR:^[[0m -inset ‘/scratch/global/lbohorquez/PCASL_analysis/results_noMask/temp/HighvsLowICG_noCov_noMask.0015.nii’: failure to open dataset
    ^[[7m** ERROR:^[[0m -inset ‘/scratch/global/lbohorquez/PCASL_analysis/results_noMask/temp/HighvsLowICG_noCov_noMask.0016.nii’: failure to open dataset
    ^[[7m** ERROR:^[[0m -inset ‘/scratch/global/lbohorquez/PCASL_analysis/results_noMask/temp/HighvsLowICG_noCov_noMask.0017.nii’: failure to open dataset
    ^[[7m** ERROR:^[[0m -inset ‘/scratch/global/lbohorquez/PCASL_analysis/results_noMask/temp/HighvsLowICG_noCov_noMask.0018.nii’: failure to open dataset
    ^[[7m** ERROR:^[[0m -inset ‘/scratch/global/lbohorquez/PCASL_analysis/results_noMask/temp/HighvsLowICG_noCov_noMask.0019.nii’: failure to open dataset
    ^[[7m** ERROR:^[[0m -inset ‘/scratch/global/lbohorquez/PCASL_analysis/results_noMask/temp/HighvsLowICG_noCov_noMask.0020.nii’: failure to open dataset
    ^[[7m** ERROR:^[[0m -inset ‘/scratch/global/lbohorquez/PCASL_analysis/results_noMask/temp/HighvsLowICG_noCov_noMask.0021.nii’: failure to open dataset
    ^[[7m** FATAL ERROR:^[[0m can’t continue after above -inset problems
    ** Program compile date = Dec 27 2019
    ^[[7m** FATAL ERROR:^[[0m ===== 3dClustSim command failed :-((( =====
    ** Program compile date = Dec 27 2019
    ~
2

The problem declares itself cryptically in the line below:

The word “Killed” is the laconic way that Unix says “I had to kill this program because it did something bad”. In almost every case, “something bad” means that the program needed too much memory to run. So the intermediate output files failed to be written, and so 3dClustSim itself could not run.

There is no simple fix for this, except to re-run on a computer system with more memory. However, I also notice that you did not use the “-mask” option to 3dttest++. You really should do that for brain imaging, to restrict the analysis to the brain region – this will give cleaner statistical results, and will also use less memory. That may help with the 3dClustSim stuff.