When I run this 3dttest++ script I am continuously met with this error in the final 2 .sdat files. All other temporary .sdat files are created/recognized with no issue. The fatal error is terminating my script before I get any cluster table thresholding files, I am only getting the minmax .1D files and the neuro output files
input:
#RUN THIS COMMAND FOR MASK BEFORE CLUSTSIM CALCUALTOR if using ROI approach
3dcalc -a $groupdata/3dMVM.FBxAggPM.45.092424+tlrc -b $signal/SEAT_n45_Grp90+tlrc -c $ROI -expr '(a*b*c)' -prefix $signal/SEAT.PMAgg.Grp90.$region
cd $results
3dttest++ -ClustSim \
-toz \
-prefix "3dttest.paired.FBxAggPM.n45.$region" \
-mask $signal/SEAT.PMAgg.Grp90.$region+tlrc -setA FBxAggPM_UnpMean \
s305 $subjdata/s305/s305.results.SEATAggPM.1mm/stats.s305+tlrc'[FB.UnpMean#1_Coef]' \
-setB FBxAggPM_Mean \
s305 $subjdata/s305/s305.results.SEATAggPM.1mm/stats.s305+tlrc'[FB.Mean#1_Coef]' \
tar -cvzf $results/*.BRIK
chmod -R 777 $results/*
error: + 3dttest++ ===== starting 3dClustSim A: elapsed = 18.9 s
12345++ t-test randomsign/permute:++ Call to setup_permute(45,45) with p_nxy=0
01672839.4051627839.4051672++ 3dClustSim: AFNI version=AFNI_24.1.19 (Jun 14 2024) [64-bit]
++ Authored by: RW Cox and BD Ward
++ Loading -insdat datasets
8349.5061728** ERROR: open_Xdataset: can't find ANY data in file './3dttest.paired.FBxAggPM.n45.amygdala.0126.sdat'
** ERROR: open_Xdataset: can't find ANY data in file './3dttest.paired.FBxAggPM.n45.amygdala.0127.sdat'
** FATAL ERROR: Cannot continue after above errors
** Program compile date = Jun 14 2024
39.** ERROR: ===== 3dClustSim command failed to output 3drefit stub :-((( =====
++ 3dttest++ ----- Cleaning up intermediate files:
04152++ loading -setB datasets
637485!
++ saving main effect t-stat MIN/MAX values in ./3dttest.paired.FBxAggPM.n45.amygdala.0125.minmax.1D
++ output short-ized file ./3dttest.paired.FBxAggPM.n45.amygdala.0125.sdat with 158 values for each of 79 volumes
======= end of .sdat output run of 3dttest++ =======
6789.012345 + 3dttest++ =============== -ClustSim work is finished :) ===============
++ ----- 3dttest++ says so long, farewell, and happy trails to you :) -----
6789.012tar -cvzf /data/STUDIES/SEAT/fMRI/derivatives/GroupLvlAnalyses/ClustSim/3dttest.paired.FBxAggPM.n45.amygdala+tlrc.BRIK
34tar: Cowardly refusing to create an empty archive
Try 'tar --help' or 'tar --usage' for more information.
5tur50860@CLA19664:/data/STUDIES/SEAT/fMRI/code/GroupLvlAnalyses$ 6789.0123456789.01++ t-test randomsign/permute:++ Call to setup_permute(45,45) with p_nxy=0
0213243546576879.809.1021324354657687!
++ saving main effect t-stat MIN/MAX values in ./3dttest.paired.FBxAggPM.n45.amygdala.0126.minmax.1D
8++ output short-ized file ./3dttest.paired.FBxAggPM.n45.amygdala.0126.sdat with 158 values for each of 79 volumes
** ERROR: Output error? file size = 0 num volumes = 0 ???
======= end of .sdat output run of 3dttest++ =======
9.0123456789.0123456789.0123456789.0123456789.0123456789.012345678!
++ saving main effect t-stat MIN/MAX values in ./3dttest.paired.FBxAggPM.n45.amygdala.0127.minmax.1D
++ output short-ized file ./3dttest.paired.FBxAggPM.n45.amygdala.0127.sdat with 158 values for each of 79 volumes
** ERROR: Output error? file size = 0 num volumes = 0 ???
======= end of .sdat output run of 3dttest++ =======
it almost looks like the tests stop before these last 2 .sdat files, goes into the permutations, fails, removes the temporary files, then tries to do the test on the outstanding .sdat files?
Any advice would be excellent, I've read the other posts on 3dClustsim particularly in 3dttest++, but I'm unsure the issues in these apply to my own.
Thanks so much
By the look of it, you are only provided 1 subject in each of setA and setB, is that right? 3dttest++ won't be happy with that. I also get different text out when I try to run a similar command on my computer (using different datasets).
Thanks for the reply!
No, I simply deleted all the other subjects from the code I shared here. Each set has 45 participants, they are listed in the same order, and they reference the same file (just different stats sub-bricks)
OK, thanks for the clarification. I ran a similar command on my computer using Bootcamp data, and it worked OK.
I notice that you have to run chmod on the results directory---are there weird permissions there? Is that preventing the *.sdat files from either being written or re-read? (Also, from the syntax there, it looks like a cd is missing from the copy+pasted code, because 3dtteste++ is run in the results/ directory itself, but the tar and chmod commands refer to ${results} as if it were just outside it?)
If you run that same 3dttest++ command in a new directory, but then don't change permissions, what is the output of:
ls -ltr
there?
Also, I notice you are zipping BRIK files manually. That shouldn't be necessary. What version of AFNI are you using? It is now the default to zip BRIK files, unless you explicitly turn it off with an environment variable. It behaves as if you had set:
Thanks for the tip on file zipping, I will check that flag now.
I run the chmod command on most of my scripts because I work on afni out of a remote server with various users. I am happy to share the things you asked for but I think showing the beginning of the script/variable setting will help.
######################################################
# SPECIFY PATHS TO FIRST LEVEL RESULTS
######################################################
# Specify the top-most directory associated with the study
set topdir = /data/STUDIES/SEAT/fMRI
# Specify the directory where subjects' first level neuroimaging data are located
set subjdata = $topdir/derivatives/IndvlLvlAnalyses/PM
# Specify the directory where subjects' group level neuroimaging data are located
set groupdata = $topdir/derivatives/GroupLvlAnalyses
# Specify the directory where the Masks will be found (90% signal, Anatomical)
set masks = $topdir/Masks
set anatomical = $masks/Anatomical
set signal = $masks/SubjMask
# Specify the directory where the group analysis results will be written
set results = $topdir/derivatives/GroupLvlAnalyses/ClustSim
set subjects = ( s305 s318 s563 s651 s747 s1146 s1521 s1541 s1562 s1571 s1577 s1585 s1587 s1610 s1614 s1622 s1629 s1639 s1671 s1692 s1693 s1696 s1712 s1714 s1718 s1722 s1746 s1754 s4075 s4084 s4089 s4096 s4111 s4128 s4136 s4148 s4164 s4201 s4207 s4209 s4211 s4228 s4262 s4266 s4273 )
# Why are we not using the blur estimate parameters from participants' [subj].blur.errts.1D files like in LEARN?
# Because the ClustSim flag in 3dttest++ (updated version of standalone 3dClustSim command) uses alternative resampling parameters for global clust-level thresholding and FPR control (false positive rate)
# 90% signal mask creation
# Copy subjects' individual masks into the individual mask directory
# foreach subj ( $subjects )
# echo ${subj}
# cd ${subjdata}/${subj}/${subj}.results.SEATAggPM.1mm/
# 3dcopy mask_epi_anat.$subj+tlrc \
# $signal/mask_epi_anat.$subj
# end
#Create 90% signal mask
#3dmask_tool -input $signal/mask_epi_anat.s*+tlrc.HEAD -prefix $signal/SEAT_n45_Grp90 -frac .90
# Specify the ROI mask you would like to explore
set ROI = $anatomical/amygdala1_lr_resampled_harvardoxford.nii
# Specify the name of the mask
set region = amygdala
# amygdala1_lr_resampled_harvardoxford.nii
# dACC2_lr_resampled_AAL3.nii
# dlPFC_lr_resampled_AAL3.nii
# insula_lr_resampled_AAL3.nii
# mPFC_lr_resampled_AAL3.nii
# TPJa_r_resampled_Mars.nii
# TPJipl_r_resampled_Mars.nii
# TPJp_r_resampled_Mars.nii
# TPJ_r_resampled_Mars.nii
# vlPFC_lr_resampled_AAL3.nii
# vmPFC_lr_resampled_AAL3.nii
# vstriatum_lr_resampled_ogi.nii
#RUN THIS COMMAND FOR MASK BEFORE CLUSTSIM CALCUALTOR if using ROI approach
3dcalc -a $groupdata/3dMVM.FBxAggPM.45.092424+tlrc -b $signal/SEAT_n45_Grp90+tlrc -c $ROI -expr '(a*b*c)' -prefix $signal/SEAT.PMAgg.Grp90.$region
cd $results
############################################################################################
# THE ANALYSIS SCRIPT BEGINS HERE
############################################################################################
3dttest++ -ClustSim \
-toz \
-prefix "3dttest.paired.FBxAggPM.n45.$region" \
-mask $signal/SEAT.PMAgg.Grp90.$region+tlrc -setA FBxAggPM_UnpMean \
s305 $subjdata/s305/s305.results.SEATAggPM.1mm/stats.s305+tlrc'[FB.UnpMean#1_Coef]' \
-setB FBxAggPM_Mean \
s305 $subjdata/s305/s305.results.SEATAggPM.1mm/stats.s305+tlrc'[FB.Mean#1_Coef]' \
tar -cvzf $results/*.BRIK
chmod -R 777 $results/*
Before the start of the t-tests for .sdat 0003, .sdat 0002 is not explicitly being saved as a 'short-ized file ...'
When doing this for .sdat 0003 it is being saved to .sdat 0000. There is no explicit output for .sdat 0001 or 0000. This may explain the fact that the last 2 .sdat files are 'empty'? But it may just be something that is supposed to happen that I don't fully understand, I'm still very new to AFNI.
I should mention that all .sdat files (that I can't open but I can see the size of) are present in the derivatives folder and are the same size 25KB.
Right right. Do you think it has anything to do with the .sdat files not being written to the corresponding t-test? Where t-test n is saving to .sdat file n-2? So then there would technically be "no data" in the last 2 files.
Two of the files that the shown error message complain about appear to exist. It isn't clear to me what is happening. I wonder if there is something happening earlier in the output?
It looks like you are running a tcsh script. Could you please put this at the end of your 3dttest++ command:
|& tee log_ttest.txt
... so that when it runs all the terminal text will be saved, and then please send that text file to me?
Yes, this is something I always add to the end of my tcsh scripts. Great idea! However when I went to check these text files they are completely empty? Earlier runs of the same script where I used -ClustSim only listed the following:
The
National Institute of Mental Health (NIMH) is part of the National Institutes of
Health (NIH), a component of the U.S. Department of Health and Human
Services.
