Effect size for 3dLME

AFNI Message Board
1

Dear Colleagues,

Hi. I would like a voxelwise effect size measure to report alongside (i.e., in a separate map) my eff Z of my within-subject quantitative variable from my 3dLME results. To give a little background, I am determining auditory arousal thresholds about eight times throughout the night during all-night functional MRI sleep studies (https://doi.org/10.1016/j.jneumeth.2018.09.019). Therefore, I have one within-subject factor of condition with up to eight levels (Arousal 1, Arousal 2, etc.) and one within-subject quantitative variable (the auditory arousal threshold for each arousal).

I found a nice structural MRI article that calculated such an effect size (https://doi.org/10.1093/sleep/zsac178) and included a nice reference on how to do it (http://dx.doi.org/10.1080/02664763.2016.1193725). Would anyone like to read these articles with me? Perhaps we could meet in-person or via video and discuss them.

Sincerely,

Dante

P.S.: I had to order the latter article through interlibrary loan, so I can email it to you.

2

Dante, what is the metric you used as input for 3dLME: Fisher-transformed correlation, BOLD response, or something else?

1 Like
3

Gang!

Hi, how are you?! I ran 3dLME two times. In reality, I am waiting on the fetal fat_lme_prep.py program to be birthed by Paul for the first one, but let me write in past tense for both.

For the first, I used the Fisher-transformed correlations from 3dNetCorr as input. For the second, I used the unstandardized regression coefficients from afni_proc.py (via 3dDeconvolve) from a single seed region correlation/connectivity analysis. That is, I used stats.“$label_fmri”“$region”“$regsiz”“$prepro”.nii"[1]" from the afni_proc.py results directory where the stats file was created with a single predictor entered via -regress_stim_files. See below for the gory details.

Sincerely,

Dante


afni_proc.py \
        -subj_id "${label_fmri[i]}"_"${region[i]}""${regsiz[i]}"_"${prepro[i]}" \
        -dsets "${label_fmri[i]}"_"${prepro[i]}"_tr80.nii \
        -blocks blur mask regress \
        -mask_import my_mask_label TT_N27_SSW_mask_wb_rs.nii \
        -mask_apply my_mask_label \
        -regress_censor_extern "${label_fmri[i]}"_censor_tr80.1D \
        -regress_stim_files "${label_fmri[i]}"_"${prepro[i]}"_tr80_"${region[i]}""${regsiz[i]}"_mean.1D \
        -regress_stim_labels "${region[i]}""${regsiz[i]}"_mean \
        -regress_use_stim_files \
        -regress_fout no \
        -regress_show_df_info no \
        -regress_no_fitts \
        -regress_no_ideal_sum \
        -regress_no_iresp \
        -regress_no_motion_demean \
        -regress_no_motion_deriv \
        -regress_polort 0 \
        -regress_est_blur_errts \
        -regress_opts_3dD \
            -rout \
            -nofullf_atall \
            -goforit \
        -regress_opts_CS \
            -pthr 0.10 0.05 0.02 0.01 0.005 0.002 0.001 0.0005 0.0002 0.0001 \
            -athr 0.10 0.05 0.02 0.01 \
        -html_review_style pythonic \
        -execute

        3dAFNItoNIFTI \
            -prefix "${label_fmri[i]}"_"${region[i]}""${regsiz[i]}"_"${prepro[i]}".results/stats."${label_fmri[i]}"_"${region[i]}""${regsiz[i]}"_"${prepro[i]}".nii \
            "${label_fmri[i]}"_"${region[i]}""${regsiz[i]}"_"${prepro[i]}".results/stats."${label_fmri[i]}"_"${region[i]}""${regsiz[i]}"_"${prepro[i]}"+tlrc.HEAD



3dLME \
    -prefix output_lme_ranslp_"$region""$regsiz"_"$prepro".nii \
    -jobs 8 \
    -model 'cond*AAT' \
    -qVars AAT \
    -ranEff '~1+AAT' \
    -SS_type 3 \
    -num_glt 1 \
    -logLik \
    -gltLabel 1 'AATeff' \
    -gltCode  1 'AAT : ' \
    -dataTable \
    Subj	cond	AAT	InputFile \
    s00003	aro1	70	20160713_2255_"$region""$regsiz"_"$prepro"_coef_abs.nii \
    s00003	aro2	25	20160713_2326_"$region""$regsiz"_"$prepro"_coef_abs.nii \
    s00003	aro3	35	20160714_0004_"$region""$regsiz"_"$prepro"_coef_abs.nii \
    s00003	aro4	40	20160714_0040_"$region""$regsiz"_"$prepro"_coef_abs.nii \
    s00003	aro5	70	20160714_0220_"$region""$regsiz"_"$prepro"_coef_abs.nii \
    s00006	aro1	5	20160901_2245_"$region""$regsiz"_"$prepro"_coef_abs.nii \
    s00006	aro2	0	20160901_2300_"$region""$regsiz"_"$prepro"_coef_abs.nii \
...
    s00105	aro3	15	20170413_0303_"$region""$regsiz"_"$prepro"_coef_abs.nii \
    s00105	aro4	35	20170413_0406_"$region""$regsiz"_"$prepro"_coef_abs.nii \
    s00105	aro5	0	20170413_0535_"$region""$regsiz"_"$prepro"_coef_abs.nii
4

Dante,

For your second analysis, I assume that you’re interested in the cross-brain correlation with the seed, not the scaling factors relative to the response at the second. So, would it better to use the Fisher-transformed correlation values from each subject as input for population-level analysis? That way, your effect magnitude at the population level would be directly related to correlation.

1 Like
5

Gang,

Hi again. “I assume that you’re interested in the cross-brain correlation[s] with the seed [at the subject level of the analysis].” Yes, I am interested in that.

“Not the scaling factors relative to the response at the [group level of the analysis].” I am interested in that, also. My ultimate question is: does the response during each arousal correlate with the cross-brain correlations with the seed?

“So, would it better to use the Fisher-transformed correlation values from each subject as input for population-level analysis? That way, your effect magnitude at the population level would be directly related to correlation.” I am working on updating my analyses to use Fisher-transformed correlation values.

What are your thoughts on how I might calculate and show an effect size for the group level analysis? I would very much like to hear them.

Sincerely,

Dante

6

how I might calculate and show an effect size for the group level analysis?

If you perform your population-level analysis with Fisher-transformed correlation values, you can covert them back to correlation values:

3dcalc -a Grp_Result+tlrc’[0]’ -expr ‘atan(a)’ -prefix GrpR
3dcalc -a Grp_Result+tlrc’[0]’ -expr ‘(exp(2a)-1)/(exp(2a)+1)’ -prefix GrpR

1 Like
7

Hm!

8

Gang,

Hi. This approach would work if I only have Fisher's z transformed r values in the model. Then the population-level coefficients would be in units of Fisher's z. However, I am interested in the correlation between Fisher's z transformed r values and another variable: auditory arousal threshold (AAT). This other variable is entered in the linear mixed model as a within-subject quantitative variable.

3dLME \
    -prefix output_lme_ranslp_${region}${regsiz}_${prepro}_${statin}.nii \
    -jobs 8 \
    -model 'cond*AAT' \
    -qVars AAT \
    -ranEff '~1+AAT' \
    -SS_type 3 \
    -num_glt 1 \
    -logLik \
    -gltLabel 1 'AATeff' \
    -gltCode  1 'AAT : ' \
    -dataTable \
...

Therefore, the units are no longer Fisher's z transformed r values. Going back to our discussion in February, perhaps I should have said that I am most interested in "the scaling factors relative to the response at the second," if I understand what you said then correctly.

How might I use AFNI to calculate and show a voxel-wise effect size for the population-level analysis of my within-subject quantitative variable? What do you think of the articles in my initial post?

Sincerely,

Dante

9

Gang,

Hi again. I have an update. The approach in those references is not what I want. Specifically, the method in the Jaeger et al. is for calculating R^2 of the fixed effects in linear mixed effects.

My desire is to obtain the effect size for my one within-subject quantitative fixed effects variable. The 3dLME model I use is effectively Fisher's z transformed r ~ 1 + AAT + cond + AAT by cond + (1 + AAT | subjects). The three fixed effect variables are: AAT, cond, and AAT by cond. The method in Jaeger et al. can only measure the sum of all three fixed effects variables (AAT, cond, AAT by cond) together. The effect size measure that I want is how ordinary R^2 changes when adding my one within-subject quantitative fixed effects variable (AAT) to the model (Selya et al., 2012).

Can this be implemented in 3dLME? Alternatively, can I get R^2 of the whole equation (fixed and random effects) so that I can calculate Cohen's f^2? Please say yes.

Sincerely,

Dante

References
Selya, A. S., Rose, J. S., Dierker, L. C., Hedeker, D., & Mermelstein, R. J. (2012). A practical guide to calculating Cohen's f(2), a measure of local effect size, from PROC MIXED. Frontiers in Psychology, 3, 111. https://doi.org/10.3389/fpsyg.2012.00111

10

Dante,

I want is how ordinary R^2 changes when adding my one within-subject quantitative fixed effects variable (AAT) to the model

Adding a variable to a model requires careful consideration. In your case, the relationship between 'cond' and 'AAT' introduces complications. Does 'AAT' differ across conditions? If so, extreme caution must be exercised.

The beta values combined with their precision in the output represent a form of effect size. However, I am not aware of any implementation for calculating R^2 or Cohen's f^2 based on the R packages nlme and lme4.

Gang

1 Like
11

Gang,

Hi. Thanks for this post.

Can the marginal R-square (fixed only) and conditional R-square (fixed + random) be obtained by using r.squaredGLMM(fit1.lme) from the MuMIn R package? See

https://stackoverflow.com/questions/45327217/r-squared-of-lmer-model-fit

for my source of this potentially useful information. Would adding the MuMIn R package to 3dLME's capabilities help all AFNI users and not just me in the Intramural Research Program? Yes, beta values are a form of effect size, but they are not comparable across studies.

I have cautiously considered this linear mixed effects analysis and its assumptions. I have read the linear mixed effects chapter in what I consider to be the regression/correlation Bible (Cohen et al., 2003) and performed all the corresponding exercises. I have performed your hands-on exercise

/home/picchionid/afni/CD/AFNI_data6/GroupAna_cases/4.RanSlp/LME.ran_slp.txt

for 3dLME and adapted it to my current analysis. I read the 3dLME -help output in its entirety and cautiously adapted Example 2 in my currently analysis. That is, reaction time (RT) in that example is exactly analogous to auditory arousal threshold (AAT) in my adaption of it.

Sincerely,

Dante

References
Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Random coefficient regression and multilevel models. In Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed., pp. 536-567). Mahwah, NJ: Lawrence Erlbaum Associates.

12

I don't have time working on this at the moment, but I'll try to implement it in a couple of weeks.

To improve interpretability, you may consider centering ATT for each condition, and then add

-qVarCenters 0 \

Gang

1 Like
13

Okay. Thanks.

14

Gang,

Hi. How are you? How are you coping with the chaos of the impending holidays? Based on your advice from other posts, I have switched from 3dLME to 3dLMEr, so the current thread is now regarding 3dLMEr.

Do you now have time to add the MuMIn R package to 3dLMEr's capabilities? It is my hope that this or something similar will benefit all AFNI users, and I very much need the above-described effect size measure for my manuscript. I am now finished with the figures with the exception of the effect-size figures, so there is little else that I can do without them.

Sincerely,

Dante

15

Dante,

I've incorporated the -R2 option into 3dLMEr to offer both conditional and marginal coefficients of determination (R^2), utilizing the MuMIn R package. This enhancement will be part of the upcoming official AFNI release. In the interim, you can test it by replacing your current 3dLMEr.R file in the AFNI binary directory with this temporary file. Please let me know if it functions as intended.

Gang

2 Likes
16

Gang,

Thank you. I am working on testing this.

Sincerely,

Dante