I have run several models in 3dLMEr, how do I compare them?

AFNI Message Board
1

Hello, I have run some models (with nested relations, from null model to full model) in 3dLMEr, now I need to pick up the best fitting one. But I don't know how to compare these models in AFNI. Is there any indicator just like AIC/BIC of the model? Or how can I use residuals to do comparisons?

Thanks!

2

Could you share the model specification line in the 3dLMEr script for each model? Also, explain the variable types and your research focus.

3

Hi Gang, actually, I conduct an exploratory/data-driven brain development study to capture the trajectory of developmental changes in children’s brains, with virtually no explicit experimental hypothesis. I intend to perform a series of sensitivity analyses to find a model that best fits the developmental changes in the BOLD signal.

The model is arranged as follow:
(1) Linear age model: 'Age + (1|SubID)'
(2) Quadratic age model: ‘Age + Age^2 + (1|SubID)’
(3) Main sex model : ‘Age + Sex + (1|SubID)’
(4) Interaction age × sex model: ‘Age + Sex + Age*Sex + (1|SubID)’
(5)… In a similar way, step by step we add variables of interest, like sex, from main effects to interactions.

In behavioral data, I can compare the AIC and BIC of different models using chi-square tests, or determine the interpretable validity of different models by comparing model residuals. But I am not sure if there is such a method in 3dLMEr that can help me compare multiple models to select a best fitting one?

Thanks for your reply.

4

One model would be enough: I suggest that you follow example 4 in the help of 3dMSS.

Since you didn't specify which extra variables you are referring to, it's hard for me to make concrete suggestions. In general, following a step-by-step modeling approach is not recommended, despite its popularity in statistics textbooks and the literature. While indices such as AIC and BIC may provide some reference value, it is important to let scientific understanding guide the modeling process, rather than the tail wagging the dog (allowing statistical measures to dictate it).

Gang