3dLMEr "Quitting due to model test failure"

e0046902 · February 8, 2024, 5:46am

AFNI version info (afni -ver):

I get the "Quitting due to model test failure" when trying to run 3dLMEr with the following code:

3dLMEr -prefix "${datadir}"/second_level_SITE_wDemo_noSHAPS/"${roi}"_results/3dLMEr_results_"${roi}" \
-jobs 20 \
-resid "${datadir}"/second_level_SITE_wDemo_noSHAPS/"${roi}"_results/residuals \
-mask "${datadir}"/mask1.nii.gz \
-model 'Drug*Time+Age+Sex+(1|Subj)+(1|Site)' \
-qVars 'Age,SHAPS' \
-gltCode V5_V0 'Time : 1*Post -1*Pre' \
-gltCode Ezogabine_postVspre 'Drug : 1*Ezogabine  Time : 1*Post -1*Pre' \
-gltCode PLacebo_postVspre 'Drug : 1*Placebo  Time : 1*Post -1*Pre' \
-gltCode pl_ezo 'Drug : 1*Placebo -1*Ezogabine' \
-gltCode drug_by_time 'Drug : 1*Ezogabine -1*Placebo  Time : 1*Post -1*Pre' \
-dataTable @model_variables_${roi}.txt

The first few lines of output from 3dLMEr when checking the dataTable is:

***** Summary information of data structure *****
80 response values
2 levels for factor Time : Post Pre 
2 levels for factor Sex : Female Male 
80 centered values for numeric variable Age : 12.7 12.7 -7.3 -7.3 12.7 12.7 -13.3 -13.3 -13.3 -13.3 -7.3 -7.3 -8.3 -8.3 -8.3 -8.3 -4.3 -4.3 15.7 15.7 2.7 2.7 15.7 15.7 3.7 3.7 -13.3 -13.3 20.7 20.7 -18.3 -18.3 -3.3 -3.3 22.7 22.7 -15.3 -15.3 -16.3 -16.3 6.7 6.7 13.7 13.7 -19.3 -19.3 7.7 7.7 5.7 5.7 9.7 9.7 -19.3 -19.3 -14.3 -14.3 -17.3 -17.3 -13.3 -13.3 -3.3 -3.3 -5.3 -5.3 -7.3 -7.3 -6.3 -6.3 20.7 20.7 16.7 16.7 19.7 19.7 14.7 14.7 -10.3 -10.3 22.7 22.7 
2 levels for factor Drug : Ezogabine Placebo 
80 centered values for numeric variable SHAPS : 9.675 -12.325 -3.325 -2.325 10.675 -9.325 0.675 -5.325 1.675 -9.325 10.675 7.675 12.675 0.675 -10.325 -18.325 -0.325 -4.325 9.675 -1.325 3.675 -18.325 4.675 -8.325 -0.325 -13.325 14.675 -10.325 4.675 -2.325 -0.325 -0.325 4.675 21.675 -4.325 -4.325 13.675 -6.325 -3.325 -4.325 -7.325 -3.325 3.675 -9.325 0.675 -3.325 -3.325 -1.325 14.675 -18.325 -4.325 -3.325 -5.325 -5.325 1.675 -3.325 19.675 20.675 6.675 -12.325 9.675 3.675 6.675 -13.325 2.675 -1.325 6.675 -8.325 2.675 -0.325 -2.325 -1.325 -8.325 -4.325 -1.325 -8.325 2.675 23.675 11.675 12.675 
2 levels for factor Site : Baylor Sinai

Any idea how to resolve this?

Gang · February 8, 2024, 8:41pm

Given the limited number of sites (only two), estimating cross-site variability becomes impractical. I recommend updating the model specification for a more robust analysis. Instead of the current specification:

-model 'Drug*Time+Age+Sex+(1|Subj)+(1|Site)' \

Consider using either of the following alternatives:

-model 'Drug*Time+Age+Sex+Site+(1|Subj)' \

or

-model 'Drug*Time*Site+Age+Sex+(1|Subj)' \

Gang Chen

e0046902 · February 8, 2024, 9:17pm

Thank you @Gang . I tried removing SITE altogether but still get the model test error.

Gang · February 8, 2024, 10:49pm

Could you provide the contingency tables for Drug, Time, and Sex? Additionally, remove SHAPS from the following line as it is not part of your model:

-qVars 'Age,SHAPS' \

Gang Chen

e0046902 · February 12, 2024, 2:22am

@Gang It is very odd because I have used the exact same script before and it worked without errors.

afni/24.0.05
R/4.3.0

3dLMEr -prefix "${datadir}"/second_level_SITE_noDemo_noSHAPS/"${roi}"_results/3dLMER_results_"${roi}" \
-jobs 20 \
-mask "${datadir}"/mask1.nii.gz \
-model 'Drug*Time+(1|Subj)+(1|Site)' \
-qVars Age \
-gltCode V5_V0 'Time : 1*Post -1*Pre' \
-gltCode Ezogabine_postVspre 'Drug : 1*Ezogabine Time : 1*Post -1*Pre' \
-gltCode PLacebo_postVspre 'Drug : 1*Placebo Time : 1*Post -1*Pre' \
-gltCode pl_ezo 'Drug : 1*Placebo -1*Ezogabine' \
-gltCode drug_by_time 'Drug : 1*Ezogabine -1*Placebo Time : 1*Post -1*Pre' \
-dataTable @model_variables_${roi}.txt

the output before model error is:

Checking dataTable file:
model_variables_nacc.txt

++Good: Table is regular and rectangular.

Dimensions: 
rows: 80 | columns: 10

Data summary: 
 Variable  Detected_Type Details                           
 Subj      Subjects      Num Subjects=40                   
 Dummy     Categorical   Counts: Test=80                   
 Dummy.1   Categorical   Counts: Test=80                   
 Time      Categorical   Counts: Post=40 | Pre=40          
 Sex       Categorical   Counts: Female=38 | Male=42       
 Age       Quantitative  Min=21 | Max=63                   
 Drug      Categorical   Counts: Ezogabine=38 | Placebo=42 
 SHAPS     Quantitative  Min=14 | Max=56 | Num outliers=3  
 Site      Categorical   Counts: Baylor=40 | Sinai=40      
 InputFile Data          Number of InputFiles=80           

++ Good: All InputFiles exist.

++ Good: All InputFiles have exactly 1 volume.

++ Good: All InputFiles are on the same grid.


++++++++++++++++++++++++++++++++++++++++++++++++++++
***** Summary information of data structure *****
80 response values
1 levels for factor Dummy : Test 
2 levels for factor Time : Post Pre 
2 levels for factor Sex : Female Male 
80 centered values for numeric variable Age : 12.7 12.7 -7.3 -7.3 12.7 12.7 -13.3 -13.3 -13.3 -13.3 -7.3 -7.3 -8.3 -8.3 -8.3 -8.3 -4.3 -4.3 15.7 15.7 2.7 2.7 15.7 15.7 3.7 3.7 -13.3 -13.3 20.7 20.7 -18.3 -18.3 -3.3 -3.3 22.7 22.7 -15.3 -15.3 -16.3 -16.3 6.7 6.7 13.7 13.7 -19.3 -19.3 7.7 7.7 5.7 5.7 9.7 9.7 -19.3 -19.3 -14.3 -14.3 -17.3 -17.3 -13.3 -13.3 -3.3 -3.3 -5.3 -5.3 -7.3 -7.3 -6.3 -6.3 20.7 20.7 16.7 16.7 19.7 19.7 14.7 14.7 -10.3 -10.3 22.7 22.7 
2 levels for factor Drug : Ezogabine Placebo 
31 levels for factor SHAPS : 14 19 20 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 39 40 42 43 44 45 46 47 52 53 54 56 
2 levels for factor Site : Baylor Sinai 

Contingency tables of subject distributions among the categorical variables:

***** End of data structure information *****
++++++++++++++++++++++++++++++++++++++++++++++++++++

Reading input files now...

Reading input files for effect estimates: Done!


Range of input data: [-0.968, 1.707]

If the program hangs here for more than, for example, half an hour,
kill the process because the model specification or something else
is likely inappropriate.

Package phia loaded successfully!

~~~~~~~~~~~~~~~~~~~ Model test failed  ~~~~~~~~~~~~~~~~~~~
Possible reasons:

0) Make sure that R package lmerTest has been installed. See the 3dLME
help documentation for more details.

1) Inappropriate model specification with options -model, or -qVars.

2) In correct specifications for random effect with -ranEff.

3) Mistakes in data table. Check the data structure shown above, and verify
whether there are any inconsistencies.

4) Inconsistent variable names which are case sensitive. For example, factor
named Scanner in model specification and then listed as scanner in the table hader
would cause grief for 3dLMEr.

Also, 3dlmer does not produce contingency table output (like 3dlme) for some reason. In the summary section, Subj is not shown as a variable that is identified. Is this intended?

Gang · February 13, 2024, 2:23am

Are you still estimating the cross-site variability, as indicated by the code snippet below?

-model 'Drug*Time+(1|Subj)+(1|Site)' \

I recommend adjusting the specification to:

-model 'Drug*Time+(1|Subj)' \

Gang Chen

e0046902 · February 13, 2024, 3:22am

@Gang I removed "Site", but still get the same error. The same table input works with 3dlme.

Gang · February 13, 2024, 1:33pm

It's possible that you might be missing some R packages. To address this, run the following command in your terminal:

rPkgsInstall -pkgs 'lme4,lmerTest' -site 'http://cloud.r-project.org'

Then, rerun your 3dLMEr script. Hope this would help resolve the issue.

Gang Chen

e0046902 · February 13, 2024, 3:09pm

@Gang Unfortunately I still get the same model test failure error. Any next steps?

e0046902 · February 18, 2024, 3:59am

Thanks @Gang for helping overcome the initial error.

I am trying to test the model with:

Drug*Time + (1|Subj) + (1|Time:Subj)

But whenever I include the (1|Time:Subj) term, the model test failure shows again. Any ideas why?

Gang · February 18, 2024, 3:41pm

I assume that between the two population-level variables, Drug is a between-individual factor, while Time is a within-individual factor. In this scenario, including (1|Time:Subj) in your model specification is redundant, as it corresponds to the residuals. Explicitly incorporating it introduces a collinearity issue. For a more in-depth discussion on model specification in 3dLMEr, refer to this blog post.

Gang Chen