In our particular case, our study is rather small, so there’s a good chance that some interesting effects are below significancy threshold. So based on Gangs suggestions, we think it would be good to also report trends that are below cluster corrected significany.
Therefore, we are wondering what would be good practice for reporting results and trends of a 3dlmer analysis?
Since 3dlmer does voxelwise analysis, I would argue that unlike in bayesian analysis, with 3dlmer it is not possible to list ALL results, as the whole brain/ mask would be included at some point.
When 3dclustsim suggests f.e. a cluster size of >=50 at p=0.001 is a significant result, there seem to be two ways of searching for trends: increasing p and decreasing cluster size. When aiming to present results in a countinuous manner, it seems that we have to make a cut-off at some point, or it would be too many small and rather insignificant clusters.
What is the best approach to check/report trends from a 3dlmer model? (And therefore more complete results, even though not full results)
Adjust p value, 2. decrease cluster size, 3. do both, 4. use 3dclustsim with a different alpha error setting, or 5. something else??
it sounds like following this advice seems to be good for 3dlmer, too. Is that correct?
In my opinion, the reporting suggestions discussed in that preprint do apply to most scenarios in neuroimaging, regardless of the modeling framework.
The massively univariate modeling approach (i.e., voxel-wise analysis) assumes that the signal follows a uniform distribution across the brain, which is far from the reality; that is, a Gaussian distribution would be more reasonable. Therefore, the cluster-based adjustment for the multiple testing issue as a post hoc band-aid would not be able to fully compensate for the information loss due to the unrealistic distribution assumption. In other words, the cluster-based inference is likely excessively conservative.
What is the best approach to check/report trends from a 3dlmer model? (And therefore more complete results, even though not full results)
Adjust p value, 2. decrease cluster size, 3. do both, 4. use 3dclustsim with a different alpha error setting, or 5. something else??
There is no best approach, but some of them are practically more reasonable. All of what you mentioned sounds reasonable to me, but I will add one more for you to play with: stick to whatever the current cluster-based method provides because that’s the mainstream dogma, and then gradually fade away the rest in light of the statistical evidence strength as shown in Fig. 1F of the preprint https://www.biorxiv.org/content/10.1101/2021.05.09.443246v1 and Figs.8-9 of the preprint https://www.biorxiv.org/content/10.1101/2021.07.15.452548v1 . This can be done by tweaking the translucency through the alpha/beta values (A and B above the threshold bar) on the AFNI GUI. Maybe there is some guideline for doing this…
The
National Institute of Mental Health (NIMH) is part of the National Institutes of
Health (NIH), a component of the U.S. Department of Health and Human
Services.