3dMVM F test and t test output

[Adury]

Hello!
I am running a script with 3dMVM command for group-level analysis between 2 groups (CT, KI). I have 3 between-subject variables (Group, Sex, Age) and 1 within-subject variable (Scan).

I was expecting the output for group difference from F-test and t-test would come out the same as there 2 groups, but the output came different. After clusterization with 6 voxels at p<0.05, there are 11 clusters for F-test and 10 for the t-test (attached picture). My question is why the F-test and t-test came different?

Code:
3dMVM -prefix MVM_emx_baseline_splitBLOCK_E_Cov_AgenSex.nii -jobs 28
-bsVars ‘GroupSex+GroupAge’
-wsVars ‘Scan’
-qVars ‘Age’
-num_glt 3
-gltLabel 1 CT-KI -gltCode 1 ‘Group : 1CT -1KI’
-gltLabel 2 Group_by_Scan -gltCode 2 ‘Group : 1CT -1KI Scan : 1th1 -1th2’
-gltLabel 3 th1-th2 -gltCode 3 ‘Scan : 1th1 -1th2’ \

[Gang]

why the F-test and t-test came different?

You can verify this by checking whether the squared t-value is about the same as the F-square.

there are 11 clusters for F-test and 10 for the t-test

Find out why that extra cluster failed to survive the t-test. Possible reasons: 1) approximation, 2) one-sided vs two-sided. Change your cluster or p-value threshold, and see what happens. This also shows the intrinsic problem with artificial dichotomization. Instead, a highlight-but-not-hide approach is recommended: https://www.biorxiv.org/content/10.1101/2022.10.26.513929v2

[ptaylor]

I might have thought the 1-sided vs 2-sided (or, for clustering, like “bi-sided”) might be the issue, but it looks like the missing cluster had the same size as several others (6 voxels). Juuuust checking that there wasn’t one cluster that would be seen by scrolling down in the cluster report? Also, knowing the threshold in each case would be useful (did you provide a p-value and use the internal p-to-stat calculation for each dataset?).

I notice the the peak-voxel locations are slightly different in some cases, which is slightly odd if these values should be exactly the same.

Do you have the clusterize settings you were using for each case? Note that in the cluster report of which you showed images, you can click the “3dclust” button above the table and output the 3dClusterize command being used internally—could you copy and paste those?

thanks,
pt

[Adury]

Hi Gang and Taylor,

Thank you so much for your reply and suggestions.

I have rerun the code with a small change. Instead of this
-bsVars ‘GroupSex+GroupAge’
I have used this option under the 3dMVM command
-bsVars ‘Group*Sex+Age’ \ ---- this is now giving me the same result for F-test and t-test. In terms of syntax, what is the difference between these 2 options in the 3dMVM command?

Only issue with the outcome is that- the x,y,z positions of the clusters are different in the cluster report for the tests, though visually the image shows exactly the same clusters. I am attaching side-by-side photos for F-test and t-test. Also the 3dClust outcome is attached (both are bi-sided, F value for Pthreshold= -4.13 to 4.13 and t score for Pthreshold= -2.0322 to 2.0322. That satisfies F=t^2).

[Adury]

The pictures get cropped for some reasons. I am attaching it here again

[Adury]

Retrying photo upload

[ptaylor]

Hi-

Thanks, I thiiiink I see the issue, and I noticed this looking at the top text rows of your cluster report. And thanks for including the 3dClusterize commands.

The F-stat here should be equal to t-squared—fine. There will be a bit of rounding difference, which might affect some thresholding, but that should be minimal. In each case, AFNI should be able to translate a given p-value to an equivalent stat—more fine.

But squaring t also loses valuable sign information, which is actually used in the clustering. Note that, as is likely correct, you are using ‘bi-sided’ clustering for the t-stat data: that is, adjacent positive and negative voxels do not cluster with each other. However, your F-stat clustering cannot use that sign information, because it is positive definite (F>=0, always)—also note that the F-stat clustering should not be “bi-sided”, but instead be “1-sided”. That should account for some of the difference here, as well, I think. Therefore, there will have to be the possibility of some inherent differences between 1-sided F-stat clustering and bi-sided t-stat clustering, I think.

I’m also a little surprised that the [1]st subvolume is where the F-stat info is. What is the output of:

3dinfo -verb /Users/adjury/MVM_emx_baseline_splitBLOCK_L_Cov_AgenSex_batchomit.nii

?

–pt

[Adury]

Hi Taylor,

Thank you for your reply and explanations!

I have attached the 3dinfo outcome for the image and yes, the F test sub-bricks have +ve range. Not sure yet, why the 3dclust option gave me “bi-sided” for F-tests.

I had a basic question- for the -bsVars option in 3dMVM what is the difference between “main factorcovariate1+covariate2" vs ",main factorcovariate1+main factor*covariate2”?

[Adury]

Hey Taylor,

Another thing I wanted to check with you, I am not sure how the center of mass (COM) for a cluster is calculated, but, because of the approximation/rounding if we have some changes in F and t scores, can that change the x,y,z co-ordinates the COM of that cluster?

And regarding “2 positive negative adjacent voxels don’t cluster with each other”, if I am inferring that right, does that mean-- if we have F test score for 2 adjacent voxel 4, 4 and the same voxels have t score -2 2 (assuming all survive Pthreshold)— for F test, the voxels will cluster, but won’t cluster for t test. If that sounds okay, can I say t test is being more stringent than F test?

[ptaylor]

Hi-

OK, the bi-sided vs 2-sided is selected in the initial Clusterize GUI that pops up; in this case, it won’t have any effect on the F-stat, because it is always >=0.

I do think the difference is the t-stat is signed and clustered bi-sidedly; the F-stat is only positive, and can’t distinguish between underlying positive and negative effects. Consider in the t-stats when there is a large, suprathreshold negative cluster with a couple positive, suprathreshold voxels touching it. In the bi-sided t-stat clusterizing, the clusterized output will only be made up of the negative voxels (in 2-sided clusterizing here, the output would be made up of both the positive and negative ones). In the “bi-sided F-stat” clusterizing—which is really just 1-sided clusterizing, because F>=0—the underlying sign differentiation is lost, so the clusterized output would be made up of all those voxels, because the t>0 and t<0 information has been lost.

To verify this, can I send you a Box link to get the statistics volume to look at?

thanks,
pt

[Adury]

Hey Paul,

I have uploaded the NIfTI file to the shared box folder. Thank you so much for taking the time to look at this.

Best,
Adury

[ptaylor]

Hi-

OK, so the issue does appear to be that: the F-stat clusterizing misses out on signedness, which the bi-sided clusterizing with the t-stat uses. Running 2-sided clusterizing with the t-stat gave the same results as the F-stat clusterizing.

Here were the 3 clusterize commands via the command line (and I use the 1sided ->right tail clusterizing on the F-stat, which would be the same as 2sided, because the stat is positive definite), with output cluster maps of ROIs in each case, as well as the thresholded data subvolumes:

# 1-sided F-stat
3dClusterize                                                                 \
    -nosum                                                                   \
    -1Dformat                                                                \
    -inset       MVM_emx_baseline_splitBLOCK_L_Cov_AgenSex_batchomit.nii     \
    -idat        1                                                           \
    -ithr        1                                                           \
    -NN          3                                                           \
    -clust_nvox  6                                                           \
    -pref_map    clust_map_F_1sided.nii.gz                                   \
    -pref_dat    clust_dat_F_1sided.nii.gz                                   \
    -1sided      RIGHT 4.13

# bi-sided t-stat
3dClusterize                                                                 \
    -nosum                                                                   \
    -1Dformat                                                                \
    -inset       MVM_emx_baseline_splitBLOCK_L_Cov_AgenSex_batchomit.nii     \
    -idat        10                                                          \
    -ithr        11                                                          \
    -NN          3                                                           \
    -clust_nvox  6                                                           \
    -pref_map    clust_map_t_bisided.nii.gz                                  \
    -pref_dat    clust_dat_t_bisided.nii.gz                                  \
    -bisided     -2.0322 2.0322

# 2-sided t-stat
3dClusterize                                                                 \
    -nosum                                                                   \
    -1Dformat                                                                \
    -inset       MVM_emx_baseline_splitBLOCK_L_Cov_AgenSex_batchomit.nii     \
    -idat        10                                                          \
    -ithr        11                                                          \
    -NN          3                                                           \
    -clust_nvox  6                                                           \
    -pref_map    clust_map_t_2sided.nii.gz                                   \
    -pref_dat    clust_dat_t_2sided.nii.gz                                   \
    -2sided      -2.0322 2.0322

I compared the F-stat cluster maps with each of the t-stat ones:

• F-stat clusters are different than bi-sided t-stat clusters

$ 3dDiff -a clust_map_F_1sided.nii.gz -b clust_map_t_bisided.nii.gz
++ Images differ: 84 of 53248 elements differ ( 0.16%)

• F-stat clusters are the same as 2-sided t-stat clusters

$ 3dDiff -a clust_map_F_1sided.nii.gz -b clust_map_t_2sided.nii.gz
++ Images do NOT differ

In the end, using the bi-sided t-stat clusters seems most appropriate. The loss of sign information with the F-stat comparison means that is a poorer check. The fact that the 2-sided t-stat matches the F-stat one just means that the interpretation of what is happening seems consistent (-> that the difference is due to ignoring signs in the data).

–pt

[Adury]

Hi pt,

Thank you so much for clarifying and explaining! Though I will be focusing more on bisided t-test, I just have one more question- the F-test cluster and 2-sided t-test cluster positions are not exacting matchings. What could be the reason for this?

[ptaylor]

https://afni.nimh.nih.gov/afni/community/board/read.php?1,169506,169588#REPLY

Hi-

Note that the cluster maps themselves are identical, as given by the 3dDiff command above, which is good.

The only values in the table that appear to differ relate to using the values themselves; most/all of these would be expected to be different:
+

CM values use the absolute value of the voxel values as weights.

… so any of the CM columns will likely differ.

• the Mean, SEM and Max Int values will necessarily differ (because those are different datasets).
• “MI RL” is this:

MI RL        : Coordinate of the Maximum Intensity value in the
                    Right-Left direction of the volume cluster

So, these differing means that the location of the ‘peak intensity’ in the F-stat and in the t-stat differs. I am slightly surprised by that, actually, and perhaps Gang can weigh in if that makes sense.

–pt

[ptaylor]

… and as a follow-up, Gang provided a sage observation here to solve my conundrum about the third set of differences:
There are two subvolumes that can be used here: the “-idat …” and the “-ithr …” ones. This is because with something like a t-test, we often have both an effect estimate and an associated statistic from the modeling here—in general, both should be used in reporting, see here.

Above, for the t-tstat case, the effect estimate subvolume index is 10 and its statistic is 11, so therefore we use “-idat 10” and “-ithr 11”—clusterizing is performed on the ithr subvolume, but we keep information around about data in those clusters, and it is actually the idat information that is shown in the “-pref_dat …” output. And, it is actually the idat subvolume information from which the CM RL, CM AP, CM IS, Mean SEM and Max Int, MI RL, MI AP and MI IS information comes from. And indeed, the “peak intensity” of the effect estimate can occur at a different peak location from a statistic. Therefore, one would expect many differences in those columns from just the statistic info.

In the F-stat case, the statistic does not have an accompanying effect estimate, so the ‘-idat …’ and ‘-ithr …’ subvolumes are the same thing—just the F-stat. If you changes the t-stat command so that the ‘-idat …’ and ‘-ithr …’ both loaded in just the t-stat volume (“-idat 11 -ithr 11”), then the table columns match the F-stat everywhere except for Mean, SEM and Max Int, as expected. NB: I am not recommending doing this for the t-stat in the final results you use, because the data information that accompanies the t-statistic is useful, but just for consistency checking.

I will update this information in the program help file, too, to make these things clearer there.

thanks,
pt

Additionally, the idat info is shown in a couple places in the table, too

For the F-stat case, there is no associated

So, for the t-stat case

In the 3dClusterize command for the t-stat,

[Adury]

Hi Paul,

Thank you for your elaborate reply. Yes, you can keep updating information, in the meantime I am processing the information you just provided!