Question on 3dsvm usage -thanks!!!

AFNI Message Board
1

My friend recently talked with me some questions on mvpa technique. He hopes to use 3dsvm for MVPA analysis but we both are puzzled with some options. I appreciate if anyone can offer advice here.

Our case is an event related fMRI study. Suppose that our experiment consists of 5 balanced runs. In each run of the experiment, there are four events: (1)House, (2)Building, (3)Threat Face, and (4)Scrambled Face. Each event has 45 trials. I know that we should not do the spatial smoothing for the pre-processing. Our specific discussion and question is based on 3dsvm (model training) usage, as following.

3dsvm -alpha a_Run1 \
-trainvol Run1_IM+tlrc \
-trainlabels Run1_categories.1D \
-censor Censor.1D \
-mask Mask+tlrc \
-model Model_Run1 \
-nodetrend \
-bucket Run1_fim

Questions, Run1_IM+tlrc & -trainlabels file
Some experts suggested that “use 3dDeconvolve -stim_times_IM to estimate the amplitude of the response to each individual stimulus. Create a new 3d+time BRIK from these amplitudes and then assign each one to the appropriate stimulus for 3dsvm analysis”. We have some questions.

3dDeconvolve -stim_times_IM
We find that when people use “-stim_times_IM” option, they take the assumption “fixed-shape regression”, and usually use the function “GAM, BLOCK(d,p) or BLOCK(d)”. They do not use the variable-shape regression, such as the TENT(b,c,n). Are we right? If we only can use the “fixed-shape regression” with the 3dDeconvolve -stim_times_IM", then things are complex.

When we use “3dDeconvolve - stim_times_IM” function with the assumption “fixed-shape regression”, We perhaps can get 40 trials for the event “House”, 38 trials for the event “Building” , 45 trials for the event “Threat Face”, and 41 trials for the event “Scrambled Face” (after I excluded the wrong and miss trials), for Run1. In the 3dDeconvolve output, they showed something like Bricks below (I use “3dinfo -verb” command to show),

Sub_Brick #0, “Full_Fstat”,
Sub_Brick #1-40: "House #1_Coef, House #2_Coef…House #40_Coef’.
Sub_Brick #41-79: "Building #1_Coef, Building #2_Coef…Building #38_Coef’.
Sub_Brick #80-124: "ThreatFace #1_Coef, ThreatFace #2_Coef…ThreatFace #45_Coef’.
Sub_Brick #125-165: "ScrambledFace#1_Coef, ScrambledFace #2_Coef…ScrambledFace #41_Coef’.
Sub_Brick #166, “Error#0”

As you see, they are list sequentially since 0 to 166. How should I prepare the option " -trainlabels "? Suppose the part of the sequence in a run of the experiment coding is as following (codes: House 1, Building 2, Threat Face 3, Scrambled Face 4): 2 2 1 1 2 4 3 4 4 3 1 2 4 4 3 3 3 2 1 1…

Does it mean that we just use this 1D data as the run’s trainlabels input file here? Or should weI prepare them as 1 1 1 1 1 2 2 2 2 2 3 3 3 3 3 4 4 4 4 4 …, as the Bricks in the "Run1_IM+tlrc " are sequential?

Is that MVPA only can be used to classify two conditions? Then does it mean that we only can compare one condition, against the other three conditions at one time? If we want to classify 1 and 2, then 3 and 4 should all be 999? If so they maybe 1 1 1 1 1 2 2 2 2 2 999 999 999 999 999 999 999 999 999 999? Or, as above, 2 2 1 1 2 999 999 999 999 999 1 2 999 999 999 999 999 2 1 1?

Question, MVPA based on several Regions

If we only want to focus on several special regions, such as temporal lobe (e.g, Hippocampus, parahippocampal cortex…). how can we select the features from the third (5s) or fourth (7.5s) scans? Our question goes back, can we use 3dDeconvolve -stim_times_IM with variable-shape regression? I know if I do the ROI analysis, perhaps I can use the IRF-Shape nonfixed 3dDeconvolve, such as CSPLIN (0,15,7), then select the features from 5s or 7.5s TRs.

Question, Prediction accuracy

After we use the model to predict/Testing the remained data, we can get the prediction accuracy for each subject. Can we just group all the accuracy data and do the one-tailed T test (compared to the chance level 50%) to see whether the MVPA model is good/significant or not? Some people suggested “group-level classifier accuracy was computed using a one-sampled t-test versus chance (50%) and assessed for significance with permutaiton testing”. Any good pointers?

Thank you very very much,
Juan Wu

2

Hi Juan,

I’m one of the developers and currently maintain 3dsvm in Stephen LaConte’s lab.

3dsvm was originally designed for blocked designs, but it can be used for event-related designs with some extra prep work. We are planning to add options to better accommodate event-related designs in the future.

[i]Questions, Run1_IM+tlrc & -trainlabels file:

3dDeconvolve -stim_times_IM
We find that when people use “-stim_times_IM” option, they take the assumption “fixed-shape regression”, and usually use the function “GAM, BLOCK(d,p) or BLOCK(d)”. They do not use the variable-shape regression, such as the TENT(b,c,n). Are we right? If we only can use the “fixed-shape regression” with the 3dDeconvolve -stim_times_IM", then things are complex.[/i]

De-convolving and normalizing the BOLD response first is a good idea. I haven’t tried using the TENT response model, but I think it should work as well.

[i]When we use “3dDeconvolve - stim_times_IM” function with the assumption “fixed-shape regression”, We perhaps can get 40 trials for the event “House”, 38 trials for the event “Building” , 45 trials for the event “Threat Face”, and 41 trials for the event “Scrambled Face” (after I excluded the wrong and miss trials), for Run1. In the 3dDeconvolve output, they showed something like Bricks below (I use “3dinfo -verb” command to show),

Sub_Brick #0, “Full_Fstat”,
Sub_Brick #1-40: "House #1_Coef, House #2_Coef…House #40_Coef’.
Sub_Brick #41-79: "Building #1_Coef, Building #2_Coef…Building #38_Coef’.
Sub_Brick #80-124: "ThreatFace #1_Coef, ThreatFace #2_Coef…ThreatFace #45_Coef’.
Sub_Brick #125-165: "ScrambledFace#1_Coef, ScrambledFace #2_Coef…ScrambledFace #41_Coef’.
Sub_Brick #166, “Error#0”

As you see, they are list sequentially since 0 to 166. How should I prepare the option " -trainlabels "? Suppose the part of the sequence in a run of the experiment coding is as following (codes: House 1, Building 2, Threat Face 3, Scrambled Face 4): 2 2 1 1 2 4 3 4 4 3 1 2 4 4 3 3 3 2 1 1…

Does it mean that we just use this 1D data as the run’s trainlabels input file here? Or should weI prepare them as 1 1 1 1 1 2 2 2 2 2 3 3 3 3 3 4 4 4 4 4 …, as the Bricks in the "Run1_IM+tlrc " are sequential?[/i]

[b]In this example, the file for –trainlabels should contain the following:

line 1: 9999
line 2-41: 1
line 42-80: 2
line 81-125: 3
line 126-166: 4
line 167: 9999

This excludes sub-brick #0 and #166, assigns class-1 to sub-brick #1-40, class-2 to sub-brick #41-79, class-3 to sub-brick #80-124 and class-4 to sub-brick #125-165. However, to keep the number of observations balanced across classes (i.e. equal number of observations for each class), I would randomly exclude some of the observations by using ‘9999’ or by using the -censor option.[/b]

Is that MVPA only can be used to classify two conditions? Then does it mean that we only can compare one condition, against the other three conditions at one time? If we want to classify 1 and 2, then 3 and 4 should all be 999? If so they maybe 1 1 1 1 1 2 2 2 2 2 999 999 999 999 999 999 999 999 999 999? Or, as above, 2 2 1 1 2 999 999 999 999 999 1 2 999 999 999 999 999 2 1 1?

During training, 3dsvm will automatically train a binary classifier for each class combination and only select the data (sub-bricks) belonging to that class combination (e.g. for the 1-vs-2 classifier, the sub-bricks belonging to all other classes (class-3,class-4) as well as sub-bricks labeled with 9999s will be ignored). During testing, the data will be sent through all binary classifiers and the class membership will be determined using a Directed Acyclic Graph (option: -multiclass DAG) or max votes wins (option: - multiclass vote).

[i]Question, MVPA based on several Regions

If we only want to focus on several special regions, such as temporal lobe (e.g, Hippocampus, parahippocampal cortex…). how can we select the features from the third (5s) or fourth (7.5s) scans? Our question goes back, can we use 3dDeconvolve -stim_times_IM with variable-shape regression? I know if I do the ROI analysis, perhaps I can use the IRF-Shape nonfixed 3dDeconvolve, such as CSPLIN (0,15,7), then select the features from 5s or 7.5s TRs.[/i]

Generally, this can be done using the -mask option in 3dsvm. The mask is considered part of the model and will be automatically applied to the testing data as well. You can generate a mask for specific brain regions defined in an atlas (e.g. Hippocampus, etc.) using whereami. You can also model the events in 3dDeconvolve (using onset time, shape and duration) first, and create a mask based on the goodness of fit.

[i]Question, Prediction accuracy

After we use the model to predict/Testing the remained data, we can get the prediction accuracy for each subject. Can we just group all the accuracy data and do the one-tailed T test (compared to the chance level 50%) to see whether the MVPA model is good/significant or not? Some people suggested “group-level classifier accuracy was computed using a one-sampled t-test versus chance (50%) and assessed for significance with permutaiton testing”. Any good pointers?[/i]

I think a one-tailed t-test of prediction accuracies vs. 50% can be used to check for above-chance significance at a group level. I would recommend estimating prediction accuracy (for each subject) using enough independent data (at least 20%) that was never used for training, or by cross-validation (at least 20 permutations) leaving out at least 20% of the data. Maybe someone else with a better statistics background can answer this question in more detail.

Jonathan

3

Dear Jonathan,

Thanks so much for your very patient reply. I have one further question.

“During training, 3dsvm will automatically train a binary classifier for each class combination and only select the data (sub-bricks) belonging to that class combination (e.g. for the 1-vs-2 classifier, the sub-bricks belonging to all other classes (class-3,class-4) as well as sub-bricks labeled with 9999s will be ignored). During testing, the data will be sent through all binary classifiers and the class membership will be determined using a Directed Acyclic Graph (option: -DAG) or max votes wins (option: -vote)”.

How should I define the classifier? Or should I just define this by the 1D data? For instance, I want to do the 1-vs-2 classifier, then I should list the 1D as below?

line 1: 9999
line 2-41: 1
line 42-80: 2
line 81-125: 9999
line 126-166: 9999
line 167: 9999

But according to your words “the sub-bricks belonging to all other classes (class-3,class-4) as well as sub-bricks labeled with 9999s will be ignored”, my understanding is that I do not need set the “line 81-125: 3–> 9999; line 126-166: 4–> 9999”. What is the secret?

Again, thank you very much,
Juan

4

Sure, no problem, I’m happy to help.

Yes, you can simply define the 1-vs-2 classifier using the -trainlables file and you don’t need to label the sub-bricks belonging to the other classes with 9999s to exclude them from being used to compute the 1-vs-2 classifier. However, the computation time will be much longer if you include all classes, since 3dsvm will train classifiers for all possible class combinations defined in the -trainlabels file. If you are mainly interested in the 1-vs-2 classifier and are trying to figure things out, it will be more time efficient to only train one classifier. I would also look at the -censor option. Here, you can provide a .1D file and include sub-bricks by putting a ‘1’ and exclude by putting a ‘0’ on the corresponding line.

Jonathan

5

HI Jonathan,

I do not understand your message exactly. We do not want to include all classes. Let us have an example which I described before, say I only want to do the 1 vs.2 classifer. Can you confirm my 1D file below is correct or not?

line 1: 9999
line 2-41: 1
line 42-80: 2
line 81-125: 9999
line 126-166: 9999
line 167: 9999

or it should be

line 1: 9999
line 2-41: 1
line 42-80: 2
line 81-125: 0
line 126-166: 0
line 167: 9999

I assume that the latter one is correct. Am I right?

Besides, I checked and found the option “-multiclass mctype (i.e., either DAG or vote)”. Any difference between DAG and vote?

Thanks for your time,
Juan

6

Hi Juan,

Yes, you can use this:

line 1: 9999
line 2-41: 1
line 42-80: 2
line 81-125: 9999
line 126-166: 9999
line 167: 9999

to train a single binary classifier (1-vs-2).

If you use this:

line 1: 9999
line 2-41: 1
line 42-80: 2
line 81-125: 0
line 126-166: 0
line 167: 9999

three binary classifiers will be trained and saved in the model (0-vs-1, 0-vs-2, 1-vs-2).

Hope this helps,
Jonathan

7

Hi Juan and other AFNI users
I am interested in the MVPA by 3dsvm. However, I am confused with the 3dDeconvolve step in your first question. Would you please send me the 3dDeconvolve manuscripts and input files by e-mail? That will be highly appreciated for me. My email is zharj@mail.ustc.edu.cn
Thanks.
Best.
rujing zha

8

Dear all,
I also cannot find the feature selection (which voxels and which TR should be included) parameter in the 3dsvm. Does it mean I should implement the feature selection by specifying the -censor option or/and -mask option?
Thanks.
rujing zha