Hello,
I am performing a t-test to see if my effect is lateralized as hypothesized. I do this by comparing the original contrast (Original) with the contrast in the symmetric voxel in the other hemisphere (Flipped). I used ttest++ with the ClustSim option.
I already ran a 3dClustSim simulation on averaged ACF parameteres, for the group level effect. The threshold in this table is VERY different from the ttest++ ClustSim output. Which of these is better to rely on?
I have an intuition that 3dClustSim is not the right thing to use when I compare two voxels on the same brain, but I am not sure. Any response is appreciated!
Thank you.
** EDIT **
In the aformentioned ttest++ with ClustSim, I could show that one stimulus has a lateralized effect and the other does not. I ran a second analysis, 2-by-2 ANOVA with 3dANOVA3, to identify clusteres significant for an interaction between Laterality (original/flipped) and the type of stimulus.
As I noted, the thresholds of ClustSim and 3dClustSim are far from being the same. But for 3dANOVA3 I don’t have any other option. However, I am not sure that 3dClustSim is the right thing to do when comparing original vs. flipped - the ACF function assumes that a voxel is being compared to itself, as far as I understand… I would appreciate any advice.
Hello AFNI statisticians!
I just wanted to make sure someone sees this, because I am really puzzled by this issue and would appreciate any advice on how to determine the thershold when the model compares two difference voxels (original vs. flipped brain). Intuitively 3dClustSim is not the right thing to use, as the comparison is between different voxels with different auto-correlational structure, but I see no other option with 3dANOVA3…
Thank you in advance!
Galit
I already ran a 3dClustSim simulation on averaged ACF parameteres, for the group level effect. The threshold
in this table is VERY different from the ttest++ ClustSim output. Which of these is better to rely on?
First of all, the two clusterization approaches have slightly different underlying mechanism. If you determine the minimum cluster size using 3dClustSim with the ACF parameters, you assume the spatial relatedness in the random noise in the group analysis model can be approximated by the spatial relatedness in the individual subject model. On the other hand, the option -ClustSim in 3dtttest++ defines a minimum cluster size based on permuting the residuals in the group analysis model for 1000 times, for example.
Which one is more reliable? Both approaches, just as other typical correction methods in the field, are only approximations and overly conservative, but between the two, the second one is probably better because the cluster size is directly based on the residuals from the population model itself, instead of relying on the spatial relatedness estimates at the individual level.
for 3dANOVA3 I don’t have any other option
You do. You can still rely on the 3dClustSim results. Or, if you have enough patience, you can convert the 2 x 2 ANOVA to multiple simple t-tests. For example, the interaction between factors A and B (both with 2 levels) is essentially equivalent to the t-test (A1B1-A1B2)-(A2B1-A2B2) or (A1B1-A2B1)-(A1B2-A2B2), which can be directly handled by 3dttest++. You can similarly formulate other contrasts such as main effects for A and B.
Hi Gang,
Thank you for the suggestion! I tried running 3dttest++ on two sets of differences, unfortunately nothing came out.
The reason I am so surprised, is that for the same p and alpha thresholds, I need 225 voxels according to 3dttest++ while only 20 according to 3dClustSim. Can it be that the difference is so huge?
Galit
for the same p and alpha thresholds, I need 225 voxels according to 3dttest++ while
only 20 according to 3dClustSim. Can it be that the difference is so huge?
Indeed the difference is surprisingly large! I’m at my wits’ end and I’m not well-equipped to figure out the reason since I don’t abide by the conventional correction methods.
First of all, I’m curious to know - what are the non-convenstional methods by which you would correct for multiple comparisons in a whole brain analysis?
Regarding my 3dClustSim - it took me some time to be able to put into words why I think this is the wrong analysis to use when comparing a voxel to its symmetric voxel in the other hemisphere. This also might explain the discrepency between 3dClustSim and ttest++. Let me know if you think this makes sense:
In 3dClustSim we ask what the chances are to get a difference between two beta values in the same voxel, given only the noise in that voxel (i.e. the correlational structure of that voxel). However, when I compare two betas that come from two different voxels, which might have different baselines or different correlational structures, then there is really no sense in asking whether this difference stems from the noise in one voxel alone. For example, let’s say the coefficient in voxel A is 0.05 and in voxel B it is 0.03. Then the question whether it is really the case that 0.05-0.03>0 cannot rely only on voxel A. It could be that in voxel A a difference between 0.05 and 0.03 is significant (or not), but when taking into account the noise in voxel B then the reult flips.
Does this make sense? If so, does it mean the ttest++ is better to use, or does ttest++ also hide an assumption that betas are compared within voxels?
Your argument about handling multiplicity through 3dClustSim vs. 3dttest++ seems to be consistent with my original hypothesis: “between the two, the second one is probably better because the cluster size is directly based on the residuals from the population model itself, instead of relying on the spatial relatedness estimates at the individual level.”
The question is, if 3dttest++ demands that all clusters have to reach the cutoff of 225 voxels, is the penalty too severe?
As for the current threshold, ttest++ might be a bit too severe, but I don’t really know. For the paper, as I can’t think of a better solution, I think I will report both with reservations… Thanks again!
The
National Institute of Mental Health (NIMH) is part of the National Institutes of
Health (NIH), a component of the U.S. Department of Health and Human
Services.