I’m trying to decide whether or not to use motion parameters as regressors in a patient vs. control dataset. I ran afni_proc.py once with and once without motion regressors, and then ran gen_ss_review_table.py to compare the different metrics for each model. I found that the maximum F-stat (masked) for each case was usually larger for the model without motion regressors. However, I’m not sure if this is the best metric to use as a comparison between models - do you recommend other metrics to evaluate when trying to decide on whether or not to include motion regressors (or other aspects of the model e.g., motion censoring limits)?
The maximum F-stat is not a reliable metric for much of
anything. Maximums are by their definition, extreme. If
the max F were higher without using motion regressors, it
just suggests that at the voxels with a max F, the motion
parameters did not fit the data. Maybe those subjects
did not move much.
More to the point, one should basically always include
motion parameter regression, assuming your model is not
highly correlated with motion (if it is, that is a big
problem). The motion parameters do not cost many degrees
of freedom.
And the goal (of single subject processing) is presumably
to get accurate subject beta weights to the group analysis,
not to maximize the F at individual voxels in the single
subject analyses.
Hi Rick,
Thank you so much! This makes sense to me. Would you say it would be better then to look at the residuals as a metric of model fit (getting accurate subject beta weights)?
Also as a follow-up in a similar vein, I’m trying to decide on the best limit to set for censoring motion. I’ve tried both .3 and .9, and I’ve looked at the maximum residual value for each model (although perhaps using the maximum, as you mentioned with the F stat, is not the best way to go). For the most part a .3 limit yields a smaller maximum residual value, but of course, .3 censors a lot more data. Is there a best practice for how much censoring is too much?
The F-stat is indeed a reasonable metric for model fit
(that is its purpose). But just do not worry about the
maximum value too much, and do not generally use the
maximum to decide whether to include certain regressors.
The @ss_review_driver script generated from afni_proc.py’s
proc script finishes off by loading the final stats output
and thresholding the Full-F at 90 percentile. That should
be a reasonable starting point for whether the model fits
(that and actually viewing the fitts time series on top of
the input data).
It is hard to say what a best anything is, motion censoring
level included. The censoring level is highly dependent on
the type of subjects you have. If they are prone to motion
(or if the stimuli make it likely), a higher limit will be
necessary.
If the enorm value is 0.5 for example, it is likely the
subject moved at that time point, which leads to having
spikes in the (analyzed) time series, which make the
analysis (including resulting beta weights) less reliable.
A group analysis can actually improve with more strict
censoring, as it can mean removing more of the noise.
So if using a limit of 0.3 leaves sufficient data (e.g. when
dropping subjects with more than 30% of time points censored
out, not too many subjects are lost), then it might be good
to stick with that limit. If too many subjects are being
lost, then it is probably necessary to raise it.
rick
The
National Institute of Mental Health (NIMH) is part of the National Institutes of
Health (NIH), a component of the U.S. Department of Health and Human
Services.