Hoping to get some advice on interpreting 3 way interactions in 3dMVM. In our study, we are interested in the interaction between group (2 levels) and a continuous behavioral measure. Our primary interest is in this groupbehavior interaction, but there is a lot of recent work suggesting that sex should be included as a potential moderator, so we wanted to address it as a factor (not a covariate). Therefore, we modeled a 3-way interaction to do this (groupbehaviorsex) and got out all the component interactions. The results from the groupbehaviorsex interaction are not very robust (low p-values) likely due to relatively low power. However, for the interaction of interest (groupbehavior) we get very clear, interpretable results. There are also pretty strong effects for the groupsex interaction. I did run another, reduced model (just a 2-way interaction- groupbehavior), and found that the results for the group*behavior interaction, while overlapping with the results from the 3-way interaction model, were quite different in many respects.
I was hoping one of you could help me understand the difference between what i’m seeing in the groupbehavior interaction when sex is included as a factor (i.e. 3-way interaction model), versus what i’m seeing in the groupbehavior interaction when sex is just a covariate (i.e. 2-way interaction model). Importantly, I’m wondering if it’s appropriate to interpret the group*behavior results from the 3-way interaction model given the a priori concern about moderation by sex.
there is a lot of recent work suggesting that sex should be included as a potential moderator, so
we wanted to address it as a factor (not a covariate)
Do you mean that the effect of variable “sex” is only considered as an additive effect when you treat it as a “covariate”, as opposed to the situation of treating it as a factor when all possible interactions involving “sex” are included in the model?
The results from the groupbehaviorsex interaction are not very robust (low p-values) likely
due to relatively low power.
You mean actually “high” p-values?
the difference between what i’m seeing in the groupbehavior interaction when sex is included as
a factor (i.e. 3-way interaction model), versus what i’m seeing in the groupbehavior interaction
when sex is just a covariate (i.e. 2-way interaction model).
It’s difficult to guess without access to the real data/result, but I suspect that the difference in terms of the two-way interaction of group:behavior between the two models (the second model is nested within the first one) is caused by the three-way interaction effect. If the three-way interaction effect is present (even if marginally significant, for example, with a voxel-wise p-value of 0.1 or below), it is not surprising to have such a difference. You may consider extracting the effect estimate at a region of interest from each subject, and plot out those effects and closely examine their relationships.
To be more direct, my real question is: is it ok to focus the interpretation of my results on the groupbehavior interaction, if that effect was observed in the context of a 3-way interaction model (groupbehaviorsex)? I have extracted the clusters from the groupbehavior interaction and graphed them and everything makes good sense.
While i do get a few significant clusters for the 3-way interaction (groupbehaviorsex), none of those clusters overlap with the groupbehavior results (so in other words, the groupbehavior results are not modified by sex).
So, one thing i haven’t mentioned is that there are a lot of significant groupsex interactions, and one of these clusters overlaps with the groupbehavior results. I’ve graphed these also, and in all cases, the effect is driven by the males with mental disorder activating more than females with mental disorder- no sex differences in controls.
I agree, it’s not surprising to me that the results are different, but I’m trying to understand why they are different. Is it because the 3-way interaction model includes the groupsex interaction, which is accounting for important variance not accounted for in the two factor model (groupbehavior)?
is it ok to focus the interpretation of my results on the groupbehavior interaction, if that effect was observed in the
context of a 3-way interaction model (groupbehavior*sex)?
There is a famous argument in statistics that states “it does not make sense to interpret the main effect in the presence of interaction” (see here: http://www.theanalysisfactor.com/interpret-main-effects-interaction/). That argument can be extended to a more general situation: a main effect or an interaction can be difficult to interpret (main effect can be considered as a first-order or one-way interaction) in the presence of a higher-order interaction.
While i do get a few significant clusters for the 3-way interaction (groupbehaviorsex), none of those clusters
overlap with the groupbehavior results (so in other words, the groupbehavior results are not modified by sex).
…
I’m trying to understand why they are different. Is it because the 3-way interaction model includes the groupsex
interaction, which is accounting for important variance not accounted for in the two factor model (groupbehavior)?
Are those " few significant clusters for the 3-way interaction" after multiple testing correction or under a voxel-wise p-value of 0.1? Even with the latter situation, it’s still possible to have a scenario as you described because an effect (e.g., three-way interaction), even if statistically non-significant, may still have some impact on the significance of other effects.
Ideally you could fit a different for each voxel or region through model tuning, but that strategy is impractical with neuroimaging through the massively univariate approach.
The
National Institute of Mental Health (NIMH) is part of the National Institutes of
Health (NIH), a component of the U.S. Department of Health and Human
Services.