I guess I am running into a trivial bug related to the calculation of MME in your paper: Chen, G., Taylor, P.A., Haller, S.P., Kircanski, K., Stoddard, J., Pine, D.S., Leibenluft, E., Brotman, M.A., Cox, R.W., 2018. Intraclass correlation: Improved modeling approaches and applications for neuroimaging. Human brain mapping 39, 1187–1206.
The small bug is in the 757th line of 3dICC.R (in NIH Biowulf’s most current version of AFNI) and it is the wrong var name of “Mask”, which should be “lop$maskData” per my guess. Can you verify? Thanks.
Thanks for the confirmation.
I was just curious on using your MME method by replacing the unknown variance with some sort of weighting/precision (such as the T-map used in your example) prior info.
As for my data, they are resting fMRI, no task. I used the z-value of the Pearson correlation as the weight info to check the ICC of correlation maps.
The MME result seems meaningful (i.e., similar to LME map) but having generally greater ICC than pure LME.
Do you think this weighting is meaningful? Or should I put the inverse of z-value as the weight parameter (I am a little confused on MME method mentioned in your paper and the example usage on 3dICC)?
I used the z-value of the Pearson correlation as the weight info to check the ICC of correlation maps.
Are you using both correlation coefficients and their Z-values as input for 3dICC with the latter under the option -tStat?
Do you think this weighting is meaningful? Or should I put the inverse of z-value as the weight parameter (I am a
little confused on MME method mentioned in your paper and the example usage on 3dICC)?
Hmm… in this case I tend to think that the weighting is difficult to justify. If I understand your situation accurately, the Fisher Z-values are simply a transformed version of your original correlation coefficients. Then, you seem to be double-dipping the data: using the same data to reenforce the original information.
I used the z-value of the Pearson correlation as
the weight info to check the ICC of correlation
maps.
Are you using both correlation coefficients and
their Z-values as input for 3dICC with the latter
under the option -tStat?
Yes.
Do you think this weighting is meaningful? Or
should I put the inverse of z-value as the weight
parameter (I am a
little confused on MME method mentioned in your
paper and the example usage on 3dICC)?
Hmm… in this case I tend to think that the
weighting is difficult to justify. If I understand
your situation accurately, the Fisher Z-values are
simply a transformed version of your original
correlation coefficients. Then, you seem to be
double-dipping the data: using the same data to
reenforce the original information.
I am not sure about the double-dipping, since I am not sure if the input weight should be the sample variance estimation or its inverse.
In your paper of the MME, the input weight is assumed to replace the unknown sample error’s variance \sigma_e, which is same across all subjects and all sessions in LME (no subscript i or j, but same subscript e across all subjects (i) and all sessions (j) ).
When I put the Z-values (a non-linear transformation of Pearson correlation, not as simple as a linear mapping) as the weight into MME, with a bigger correlation value, the sample error’s variance \sigma_ij is “made” bigger since its z-score is usually higher. This bigger error variance \sigma_ij should hurt the bigger correlation value in the ICC calculation, since the error variance should be as small as 0 in the ideal case. Is my understanding right?
If I am right, then this may not reinforce the original information.
Or I may understand wrongly, the weight input should be the precision or the inverse of the \sigma_ij, and I should input the inverse of Z-values. In this case, there will be enhancement instead of ICC undermining, then this seems not appropriate.
Sorry for many questions, I am just very curious about MME and have many confusions while not put my time into reading the code to align with your paper.
The issue of how the weighting is executed in 3dICC is not really important in the current context. First, usually we would not directly feed the Pearson correlation coefficients as input for further analysis because they are bounded within [-1,1]. Instead they are converted to Fisher Z-scores so that Gaussian distribution could be conveniently applied. Second, the precision for a Pearson correlation coefficient is not readily available. There is a theoretical argument of using 1/sqrt(N-3) as its standard error, but that would likely be virtually the same across all the subjects; thus, there would be basically no weighting. More importantly, such an ad hoc standard error is probably not necessarily a good approximation in reality.
Thanks for your comments.
For your 1st comment, I totally agree. I am just playing around your MME method. I would feed Z-score in the LME as my formal ICC calculation result.
For your 2nd comment on the Z-score normalization, I do think normalizing with 1/sqrt(N-3) is a real “normalization”. Since after the Fisher Z-score conversion from Pearson correlation, this normalization would follow N(0,1). By the way, the bigger the N, the more probable the Pearson correlation being 0, thus this normalization would be helpful when N is huge. Or in another case, when there are different N across subjects.
With same N across subjects, normalizing with this stdev does not hurt since all subjects have same N.
Anyway, the discussion on 1/sqrt(N-3) is kind of off-topic on using MME
Let’s switch back to the MME.
let’s say I input Z-score as the main input. What do you think the weight input as Pearson correlation, or the P-value?
There should be one of them being hurting while another boosting. The hurting one may not be called “double dipping”.
I am not sure if the boosting one can be called “double dipping” too. I re-checked the double dipping paper by “Kriegeskorte et al., 2009”. The double dipping usually involves two steps (kind of circular), and the result may not reflect the data or just determined by the assumption. I don’t think it is the case when I play some weighing in MME.
I understand your points on how to select MME or LME in your paper. However, I don’t have task data to process with.
I still want to try MME since ANOVA/LME’s same sample error assumption can be easily violated across subjects and sessions.
Please advise if I understand wrongly. I would probably just use LME and stop messing with MME until your newer method
The discussion seems to have been drifting away from the original context. The initial issues were
** whether it is appropriate to use Pearson correlation coefficients as input
** whether it is justifiable to use the Fisher-transformed Z-scores as weights for the Pearson correlation coefficients in ICC computation
If you feed the Fisher-transformed Z-scores into 3dICC as input, I don’t see how you could use the same values as their own weights (thus the “double-dipping”). It would be a different topic if you move the target to something else (e.g., 1/sqrt(N-3)).
The
National Institute of Mental Health (NIMH) is part of the National Institutes of
Health (NIH), a component of the U.S. Department of Health and Human
Services.
