concatenating time series across runs while accounting for scanner drift and value changes

AFNI Message Board
1

Hi All,

My understanding is that during 3ddeconvolve the GLM automatically creates a baseline regression that models baseline shifts between and within runs (via the polort function). This is all incorporated to the output file: stats.subj+tlrc. After 3ddeconvolve has run my script concatenates all of the runs in my task (named: all_runs+subj+tlrc). I have four runs that were all collected during one fMRI session. I am assuming that the concatenated file (all_runs) is just that and do not include any adjustments for scanner drifts or jumps in values between scans. Is this correct?

If so, is there a way to concatenate the time series across all of the runs and adjust for scanner drift and shifts in values between runs?

If my logic is incorrect, thanks for clarifying the steps. Below is my 3ddeconvolve script for reference:

run the regression analysis

3dDeconvolve -input pb04.{$subj}.r*.scale+tlrc.HEAD
-censor motion_{$subj}_censor.1D
-polort 3
-num_stimts 17
-num_glt 12
-stim_times 1 {$subj}_FMN1_FA.txt ‘BLOCK(6,1)’
-stim_label 1 FA
-stim_times 2 {$subj}_FMN2_MA.txt ‘BLOCK(6,1)’
-stim_label 2 MA
-stim_times 3 {$subj}_FMN3_NA.txt ‘BLOCK(6,1)’
-stim_label 3 NA
-stim_times 4 {$subj}_FMN4_FW.txt ‘GAM’
-stim_label 4 FW
-stim_times 5 {$subj}_FMN5_MW.txt ‘GAM’
-stim_label 5 MW
-stim_times 6 {$subj}_FMN6_NW.txt ‘GAM’
-stim_label 6 NW
-stim_times 7 {$subj}_FMN7_FNW.txt ‘GAM’
-stim_label 7 FNW
-stim_times 8 {$subj}_FMN8_MNW.txt ‘GAM’
-stim_label 8 MNW
-stim_times 9 {$subj}_FMN9_NNW.txt ‘GAM’
-stim_label 9 NNW
-stim_times 10 {$subj}_FMN10_F.txt ‘BLOCK(3,1)’
-stim_label 10 F
-stim_times 11 {$subj}_FMN11_G.txt ‘BLOCK(4,1)’
-stim_label 11 G
-stim_file 12 motion_demean.1D’[0]’ -stim_base 12 -stim_label 12 roll
-stim_file 13 motion_demean.1D’[1]’ -stim_base 13 -stim_label 13 pitch
-stim_file 14 motion_demean.1D’[2]’ -stim_base 14 -stim_label 14 yaw
-stim_file 15 motion_demean.1D’[3]’ -stim_base 15 -stim_label 15 dS
-stim_file 16 motion_demean.1D’[4]’ -stim_base 16 -stim_label 16 dL
-stim_file 17 motion_demean.1D’[5]’ -stim_base 17 -stim_label 17 dP
-gltsym ‘SYM: FA -MA’
-glt_label 1 FA-MA
-gltsym ‘SYM: FA -NA’
-glt_label 2 FA-NA
-gltsym ‘SYM: MA -NA’
-glt_label 3 MA-NA
-gltsym ‘SYM: FW -FNW’
-glt_label 4 FW-FNW
-gltsym ‘SYM: MW -MNW’
-glt_label 5 MW-MNW
-gltsym ‘SYM: FW -MW’
-glt_label 6 FW-MW
-gltsym ‘SYM: FNW -MNW’
-glt_label 7 FNW-MNW
-gltsym ‘SYM: FW -NW’
-glt_label 8 FW-NW
-gltsym ‘SYM: FNW -NNW’
-glt_label 9 FNW-NNW
-gltsym ‘SYM: MW -NW’
-glt_label 10 MW-NW
-gltsym ‘SYM: MNW -NNW’
-glt_label 11 MNW-NNW
-gltsym ‘SYM: NW -NNW’
-glt_label 12 NW-NNW
-fout -tout -x1D X.xmat.1D -xjpeg X.jpg
-x1D_uncensored X.nocensor.xmat.1D
-fitts fitts.{$subj}
-errts errts.{$subj}
-bucket stats.{$subj}

if 3dDeconvolve fails, terminate the script

if ( $status != 0 ) then
echo ‘---------------------------------------’
echo ‘** 3dDeconvolve error, failing…’
echo ’ (consider the file 3dDeconvolve.err)’
exit
endif

display any large pairwise correlations from the X-matrix

1d_tool.py -show_cormat_warnings -infile X.xmat.1D |& tee out.cormat_warn.txt

create an all_runs dataset to match the fitts, errts, etc.

3dTcat -prefix all_runs.{$subj} pb04.{$subj}.r*.scale+tlrc.HEAD

2

Hello,

Indeed, all runs is just the input, concatenated
into a single dataset. The reason for making it
is to allow easy plotting along with the fitts.

So basically you want to create a new dataset, but
with the drifts removed? It seems like you might
want options for what pieces are removed, too. In
which case 3dSynthesize is the appropriate way to
go.

Via afni_proc.py or after the fact, run the original
regression with the addition of a -cbucket option,
which will save all (including baseline) betas in
the dataset you specify. That cbucket can be passed
to 3dSynthesize, along with the x-matrix, and you
can request it to put together any betas you want.

In this case, perhaps you want ‘allstim’, though it
would leave the data with a zero mean.

  • rick
3

Hi Rick,

Thank you for your reply. I followed your steps along with the information here: https://afni.nimh.nih.gov/SimAna . However, I had massive spikes in my “cleaned” dataset. Another user had the same problem (https://afni.nimh.nih.gov/afni/community/board/read.php?1,148918,148923#msg-148923) in which Gang suggested that this might be due to censoring out time points and thus, 3dSynethesize would not be a good option. He suggested PPI analysis.

Is there any other AFNI tool available to account for scanner drift etc., while also dealing with censored data? I am working with a software that assess whole brain task-based functional connectivity. This software needs one continuous timeseries file. I have four runs of my task-data (censored for motion).

4

If the trouble is that you would like the censored time points zeroed out (it isn’t clear what else might be better), than one option is to multiply this synthesized dataset by the censor time series using 3dcalc. That would just zero out those censored time points. Does that seem like what you want?

  • rick
5

Hi Rick,

Thanks for your reply. It may be my misunderstanding but when I followed your suggestion above after step 3, it did work. However, I was able to zero out the censored motion after step 4 and it worked:

  1. Run 3dDeconvole with the -cbucket option

    • This produces the file called: c.stats.$subj+tlrc

  2. Use the command grep ColumnLabels X.xmat.1D to find which columns to put into 3dSynthesize

  3. Use 3dSynthesize to create a dataset of effects of no interest

3dSynthesize -prefix EffectsNoInterest -matrix X.xmat.1D -cbucket cstats.{$subj}+tlrc -select 0 1 2 3 4 5 6 7 8 9 10 11

  1. Subtract effects of no interest from fitted timeseries data

3dcalc -a all_runs.{$subj}+tlrc -b EffectsNoInterest+tlrc -expr ‘a-b’ -prefix cleantimeseries_FC_{$subj}

  1. Multiple “clean” dataset by the censored motion timeseries (motion_$subj_censor.1D)

3dcalc -a motion_{$subj}censor.1D -b cleantimeseries_FC{$subj}+tlrc. -expr “a*b” -prefix final_timeseries_for_FC_{$subj}

  1. resample your data to match the anatomical data (given that you have already transformed all data to standard space).

3dresample -master anat_final.{$subj}+tlrc. -input final_timeseries_for_FC_{$subj}+tlrc. -prefix final_timeseries_for_FC_{$subj}_resampled

Does this seem correct?

Thank you for your help

6

That seems reasonable, though I am not sure exactly what your goal is.
Note that 3dTproject can project out columns of an X-matrix.

Step 6 is questionable. Why does the data need to be upsampled?

  • rick
7

Thanks, Rick.

Yes, you are right step 6 is not necessary.

The overall goal is to do some task-based functional connectivity analysis that correlates brain data to a continuous behavioral variable. . The next steps would be to extract the timeseries using an atlas with whole brain coverage. However, since I had multiple runs for my task, I had to subtract out scanner drift rate and motion.

Thanks for the heads up about 3dTproject.