Using a term twice in a regression

Hello everyone,

We recently ran an fMRI study and used AFNI for the analyses. It is a memory study whereby we are exploring the effects of drawing relative to writing. Scanning was done at retrieval during an old/new recognition test. We evaluated contrasts between conditions in two different ways:

Analysis version 1: First, we did just the simple effect contrast (brain activity when responding to Draw items vs. Write items). This resulted in a minimum cluster size threshold of 30 voxels, which allowed for 2 clusters to survive thresholding.

Analysis version 2: Then, we did a contrast whereby brain activity during false alarm responses on the recognition test ('old' responses to New items) was subtracted from each term (Draw - New vs. Write - New). However, note that this was a within-subject study with intermixed trials at retrieval, so the New value is constant within a participant, meaning the identical term is used twice in the model in this case. Effectively, we are comparing two difference scores here that each share a common baseline level of performance, if you will (i.e., the New term). This resulted in a minimum cluster size threshold of 9 voxels, which allowed for 5 clusters to survive thresholding (including the two clusters mentioned in analysis option 1).

We argued that using the latter analysis leads to reduced variance in the model and therefore when we determine minimum cluster size thresholds we get small required cluster sizes, allowing for more clusters to survive thresholding.

However, we had a reviewer raise the point that counting the New term twice in the model may underestimate the variance, going on to suggest that the improved model fit in the latter analysis type is likely spurious. They then suggested we should stick with only the simple effects (analysis version 1) and not use analysis version 2. However, they did not provide any references for this claim. In my mind, analysis version 2 is like running a t-test on difference scores, which should be valid.

Does anyone have thoughts on this debate, or papers to read relevant to this discussion?

Thanks in advance!

To clarify, are the two versions of your analysis distinguished by using the same model at the individual level with 3dDeconvolve, but extracting those two different effect estimates for population-level analysis? Additionally, it would be helpful if you could share your 3dDeconvolve script.

Gang Chen

If I'm understanding your question properly, I believe the answer is yes. We used afni_proc, so here is the code for that, and below is the code for the 3dANOVA command whereby we create the difference scores described in 'analysis option 2' when using the '-adiff' command:

afni_proc.py                                                                 \
    -subj_id                  ${subj}                                        \
    -blocks                   tshift align tlrc volreg mask blur scale regress \
    -radial_correlate_blocks  tcat volreg                                    \
    -copy_anat                ${anat_cp}                                     \
    -anat_has_skull           no                                             \
    -anat_follower            anat_w_skull anat ${anat_skull}                \
    -dsets                    ${dsets_epi}                                   \
    -tcat_remove_first_trs    ${nt_rm}                                       \
    -align_unifize_epi        local                                          \
    -align_opts_aea           -cost lpc+ZZ                                   \
                              -giant_move                                    \
                              -check_flip                                    \
    -tlrc_base                ${template}                                    \
    -tlrc_NL_warp                                                            \
    -tlrc_NL_warped_dsets     ${dsets_NL_warp}                               \
    -volreg_align_to          MIN_OUTLIER                                    \
    -volreg_align_e2a                                                        \
    -volreg_tlrc_warp                                                        \
    -volreg_compute_tsnr      yes                                            \
    -volreg_warp_dxyz         ${final_dxyz}                                  \
    -blur_size                ${blur_size}                                   \
    -mask_epi_anat            yes                                            \
    -test_stim_files          no                                             \
    -regress_stim_times       ${timing_files}                                \
    -regress_stim_labels      ${stim_classes}                                \
    -regress_basis_multi      "BLOCK(2,1)"                                   \
    -regress_motion_per_run                                                  \
    -regress_censor_motion    ${cen_motion}                                  \
    -regress_censor_outliers  ${cen_outliers}                                \
    -regress_compute_fitts                                                   \
    -regress_opts_3dD         -jobs 4                                        \
                              -num_glt 6                                     \
                              -gltsym 'SYM: draw_tar -write_tar' -glt_label 1 draw_write  \
                              -gltsym 'SYM: draw_tar -list_tar' -glt_label 2 draw_list  \
                              -gltsym 'SYM: draw_tar -new_tar' -glt_label 3 draw_new  \
                              -gltsym 'SYM: list_tar -write_tar' -glt_label 4 list_write  \
                              -gltsym 'SYM: list_tar -new_tar' -glt_label 5 list_new  \
                              -gltsym 'SYM: write_tar -new_tar' -glt_label 6 write_new  \
                              -GOFORIT 10                                    \
    -regress_3dD_stop                                                        \
    -regress_reml_exec                                                       \
    -regress_opts_reml        -GOFORIT                                       \
    -regress_make_ideal_sum   sum_ideal.1D                                   \
    -regress_est_blur_errts                                                  \
    -regress_run_clustsim     no                                             \
    -html_review_style        pythonic

# Setup 3dANOVA2 command using the gen_group_command.py interface
gen_group_command.py -command 3dANOVA2 \
-write_script 3dANOVA2_script.tcsh \
-prefix 3dANOVA2_bucket_results \
-dsets $topdir/MM**/Organized_Files/Allruns/MM**.results/stats.MM**_REML+tlrc.HEAD \
-subs_betas 'draw_new#0_Coef' 'draw_write#0_Coef' 'draw_list#0_Coef'  'list_new#0_Coef' 'list_write#0_Coef' 'write_new#0_Coef' \
-options \
    -amean 1 DN \
    -amean 2 DW \
    -amean 3 DL \
    -amean 4 LN \
    -amean 5 LW \
    -amean 6 WN \
    -adiff 1 6 DN_WN \
    -adiff 1 4 DN_LN \
    -adiff 4 6 LN_WN \
    -fa F_Stat_Condition \

Your ANOVA formulation might be overly complex. If your research hypothesis centers around the contrast between Draw items and Write items, consider the following three options:

  1. One-Sample t-Test:

    • Directly perform a one-sample t-test using 3dttest++ with draw_write from the individual level.
  2. Paired t-Test (Option 1):

    • Conduct a paired t-test using 3dttest++ with draw_tar and write_tar from the individual level.
  3. Paired t-Test (Option 2):

    • Alternatively, perform a paired t-test using 3dttest++ with draw_new and write_new from the individual level.

All three approaches should yield virtually identical results, as they are both ontologically and algebraically equivalent.

Gang Chen

Thanks for the advice, Gang! If I were to stick with the ANOVA as it stands now, is there a difference between Draw - Write vs. (Draw - New) vs. (Write - New), as the reviewer suggested? They suggested the latter case would lead to systematic underestimation of variance because I'm counting the same New term twice.

The focal effect should be the driving force behind your choice of model, not the other way around. In your case, a well-designed ANOVA model might also be appropriate and could potentially yield comparable results. However, without directly examining your data, it's difficult to say definitively why the two contrasts diverge based on your current model.

Gang Chen