3dLME linear contrast output: interpreting p-values + using FDR vs. 3dClustSim for multiple comparisons

AFNI Message Board
1

Hi folks,

I have a few questions regarding interpreting general linear contrast results from 3dLME. Briefly, I have used 3dLME to perform a VBM-style analysis of structural imaging data. My design includes 5 groups of participants, with different numbers of participants per group, and I’ve performed linear contrasts between each pair of groups.

My first question involves the critical t-statistic–I was surprised to see that it was the same for all my contrasts (corresponding to an uncorrected p<0.001), even though the sizes of the two groups differed for each contrast. I suspect that this is because each orthogonal contrast is assigned degrees of freedom=1. Can you confirm this, and is there no way to take into account group sizes when determining the critical t-stat? It seems odd to use the same threshold for two contrasts that involve different total sample sizes.

Additionally, I would appreciate your advice on multiple comparisons correction. My first inclination was to use FDR correction and threshold all my contrasts of interest to correspond to q<0.05. However, I have noticed that for my better-powered contrasts (i.e., those involving larger group sizes), the FDR correction appears very lenient. In a few cases, the t-statistic corresponding to q<0.05 is actually smaller than that corresponding to p<0.05. Do such results indicate that the FDR curves are inappropriately estimated?

As an alternative to FDR, I have opted for correction to a cluster-wise alpha level of p<0.05. To do so, I ran 3dFWHMx on the 4-D residuals from my model and then ran 3dClustSim with the spatial auto-correlation parameters to determine an appropriate combination of cluster threshold and uncorrected p-value. This is appealing in some ways–for example, it is very simple to apply the same critical t-statistic to all my contrast maps. As above, though, I worry that thresholding all my contrasts at the same voxelwise p-value will be more conservative for some contrasts than for others. If you see problems with my approach and/or would recommend alternatives, I’d appreciate the advice.

Thanks,

Jeff

2

Hi Jeff,

Can you confirm this, and is there no way to take into account group sizes when determining the critical t-stat?
It seems odd to use the same threshold for two contrasts that involve different total sample sizes.

Even though those groups have different numbers of subjects, it is assumed in the model that cross-subjects variance is the same across groups. Therefore, those cross-group comparisons share the same t-distribution (and thus the same threshold for all involving contrasts).

I have noticed that for my better-powered contrasts (i.e., those involving larger group sizes), the FDR correction
appears very lenient. In a few cases, the t-statistic corresponding to q<0.05 is actually smaller than that
corresponding to p<0.05. Do such results indicate that the FDR curves are inappropriately estimated?

It’s possible to have such a “lenient” scenario because FDR correction is sensitive to the total number of overall “surviving” voxels in the brain.

I worry that thresholding all my contrasts at the same voxelwise p-value will be more conservative for some contrasts
than for others. If you see problems with my approach and/or would recommend alternatives, I’d appreciate the advice.

No, I don’t see any particular issues with your approach.

3

Thanks for your help, Gang!

Jeff