Possible Bug: Default behavior in 3dclusterize vs. afni GUI

AFNI Message Board
1

Hi everyone,

I have analyses that were done in 3dMVM, which give me f statistics. I am trying to write a script that thresholds these maps using a p-value and a minimum cluster size, using the 3dclusterize command. I was always under the impression that I should use a 2-sided test for these f-values, because the underlying contrast could be bi-directional (i.e. exp > control or control > exp). Is this standard practice?

2-tailed seems to be the default in the afni GUI (e.g. F(1,27) = 13.61; p = 001). However, if I try to replicate this behavior in 3dclusterize by using a 2-sided test, I get the following error:

** FATAL ERROR: You are asking for multisided clustering on a single-sided stat!

Other than using a 1-sided test with p=p/2, is there a way to override this error?

Thanks,
-nick

2

Hi Nick,

The F-test is indeed a 1-tailed test. In the afni GUI, an F-stat result looks the same when viewed as a 1- or 2-sided test, rather than forcing an explicit change by the user, say. 3dClusterize is being more picky than the afni GUI because the user is already forced to specify the sidedness.

However the F-stat corresponds to a 2-sided t-test, because both positive and negative results, if applicable, would be shown in the F-test. To put it another way, the F-test for a simple contrast could not possibly show a 1-sided result of the betas, e.g., positive only, even though it is a 1-tailed test. The positive and negative betas could not be separated with an F-test.

For reference, it might be nice to verify some of the numbers. In doing so, we must keep in mind that the cdf program views F as 1-sided, and t as 2-sided. No options.

In your case, since the F-stat comes from a simple contrast, t=sqrt(F).

F(1,27) = 13.61 @ p=0.001. But p.0.001 corresponds to t(27)=sqrt(13.61) as a 2 sided test. And sqrt(13.61) = 3.689173.

cdf -t2p fift 13.61 1 27
p = 0.00100109

cdf -t2p fitt 3.689173 27
p = 0.00100109

To really do a 1-tailed t-test, the threshold would have to be different, because ‘cdf’ assumes 2-sided testing for the t-stat. The 2-sided conversion from p=0.001 shows the above t:

cdf -p2t fitt 0.001 27
t = 3.68959

However for a 1-sided test, ‘cdf’ should be fooled by using 2*p:

cdf -p2t fitt 0.002 27
t = 3.42103

Verify this in the afni GUI:

  1. threshold on the t-stat in the afni GUI

  2. right-click on the “Thr” text above the threshold slider bar, and “Set p-value” to 0.001
    ==> this should set the threshold to 3.68959

  3. now right-click again, and change “Sign” to “Pos only”
    ==> the p should change to 0.002

  4. repeat step 2: right-click and set p=0.001
    ==> this should set the t-stat threshold to 3.42103

The result of step 4 should match the faked 1-sided p-to-t conversion with cdf.

Anyway, that might be extra babbling, but it hopefully clarifies things.

  • rick
3

Is this still the case in the GUI? I've run into a similar issue where the output of 3dClusterize seems to be a lower threshold (1-tailed, RIGHT_TAIL) from the output of a 3dLMEr analysis (Chi-sq). When I manually threshold in the GUI the results looks slightly different (fewer clusters). If I change the Pos & Neg to Pos only, I don't see the p-value change, though. I'm not sure which results to trust.

4

Howdy-

The GUI uses 3dClusterize under the hood for its clusterizing. You can open the Clusterize Report ("Rpt") button, and dump out the 3dClusterize command that the AFNI GUI is using, to verify that it is the same. Make sure, for example, that you are using the same "NN" (nearest neighbor) level, the same sidedness (1sided, 2sided, bisided), same cluster-size threshold voxel count (-clust_nvox), etc.

Also, I guess there could be a small difference if your underlay dataset in the GUI is on a different grid than the overlay data which is being thresholded+clusterized. The underlay dataset determines the grid for display, so the overlay data will be inherently resampled for visualization; when running 3dClusterize from the command line, there won't be an underlay. This could produce very small differences, in theory. In the first case, the data are regridded for display and then clusterized; in the command line processing case, the data are clusterized and then if you load up the results, those are regridded at time of display. Very minor/edge-of-cluster differences could appear.

Also, what is your afni -ver?

--pt

5

Hi Paul,

Thanks for your quick response!

The NN is the same, but the GUI seems to use -2sided even when I say no to Bisided. I've also double checked with the underlay and that doesn't seem to be the problem.

When voxel thresholding a p < .001 in the GUI the Chi-sq value is 13.816. But when using 3dClusterize and specifying p=0.001 the program covers this to a Chi-sq of 12.429216. Perhaps the issue is in the conversion?

From the GUI:

3dClusterize -nosum -1Dformat \
    -inset /Users/andrewlynn/Documents/Projects/HomeMath_RSA/groupanalyses/univariate/3dLMEr_output_Longit_Format_Quantity_Linear+tlrc.HEAD \
   -idat 5 \
   -ithr 5 \
   -NN 1 \
   -clust_nvox 35 \
   -2sided -1e+09 13.816

My command

3dClusterize -overwrite \
    -orient LPI \
    -inset 3dLMEr_output_Longit_Format_Quantity_Linear+tlrc.BRIK \
    -ithr 5 \
    -idat 5 \
    -mask /Users/andrewlynn/Documents/Projects/HomeMath_RSA/ref/mask/BN_Atlas_HaskinsPeds_NL_template_2.5_mask+tlrc.HEAD \
    -1sided RIGHT_TAIL p=0.001 \
    -NN 1 \
    -clust_nvox 35 \
    -pref_map 3dLMEr_output_Longit_Format_Quantity_Linear_clust_1sided_quantitybyage_map.nii.gz -pref_dat 3dLMEr_output_Longit_Format_Quantity_Linear_clust_1sided_quantitybyage_Chi-sq.nii.gz \
    > 3dLMEr_output_Longit_Format_Quantity_Linear_clust_1sided_quantitybyage_Chi-sq_report.txt
6

Howdy-

Thanks for adding those commands. Sidenote: I reformatted them to be spaced vertically so I could see the options a bit more easily.

Well, you are correct to have raised this issue, thanks for being eagle-eyed and checking this carefully. The GUI has the correct behavior, and 3dClusterize was trying to be too fancy with sidedness and p-to-stat conversions. I have fixed this now, and we will do a build shortly so that the latest AFNI versions (24.1.01 and higher) will have the fix.

In the meantime, you can use p2dsetstat do calculate the p-to-stat conversion correctly, and the put that value in on the command line. The example usage verbosely would be like (using Bootcamp data):

p2dsetstat -inset stats.FT+tlrc.'[0]' -pval 0.001 -1sided

... which might output:

++ Found input file  : stats.FT+tlrc.[0]
++ Subbrick label    : Full_Fstat
++ p-value           : 0.00100000
++ stat type and par : Ftest(2,412)
++ sidedness         : 1sided
++ Final stat val    : 7.024879

In correct usage for this F-stat which only has positive values, asking for either a 1sided or 2sided stat from this p-value will evaluate to the same statistic value.

If you want to be more programmatic you can add -quiet to the command, and it will just spit out the final stat value, which can be assigned to a variable.

Again, thanks very much for looking so closely at the various results and for pointing this out.

--pt

7

Thank you Paul for updating this for the future! I'll use this alternative approach for now.

Andrew