OC_tedort error

AFNI Message Board
1

Hello folks,

I was happily testing the multiecho combine method, OC_tedort when I ran into an issue during the regression stage:


e[7m** ERROR:e[0m !! 3dDeconvolve: Can't run past 5 matrix warnings without '-GOFORIT 5'

which was new. I believe the only change was to apply motion regressors per run. The error itself comes from here,


++ Wrote matrix values to file X.nocensor.xmat.1D
e[7m*+ WARNING:e[0m -------------------------------------------------
e[7m*+ WARNING:e[0m Problems with the X matrix columns, listed below:
e[7m*+ WARNING:e[0m !! * Columns 107 [morts_r02[3]#0] and 208 [morts_r02[104]#0] are (nearly?) collinear!
e[7m*+ WARNING:e[0m !! * Columns 112 [morts_r02[8]#0] and 211 [morts_r02[107]#0] are (nearly?) collinear!
e[7m*+ WARNING:e[0m -------------------------------------------------

I investigated these multiecho derived orts (morts is a nice name) and they are not just nearly collinear, they are absolutely identical. This is both before and after projecting out the good components.

is it a problem with the (large) number of regressors leading to some strange wrap around? The first run did not report any problems, but had a smaller number of regressors.

Also, would this large number of regressors (203, but total timepoints = 890) be expected slow down the 3dREML voxel loop?

Thanks for the work on multiecho! Let me know if there are any particular details you would like to see.

2

To be clear, do you mean that 2 of the ‘bad’ terms
from tedana.py are identical? That is odd. The
large number of regressors is expected, since there
would be many per run.

Yes, the large number of regressors is much more
work for the regression program, and will take longer.

Can you verify whether tedana.py produced identical
components, or is that somehow coming from afni_proc.py?
If you mail me the TED directory, minus the 3-D or 4-D
data, I would be happy to take a look.

What is the output of:

1d_tool.py -infile meica_mix.1D -show_cormat_warnings

Thanks,

  • rick
3

Thanks for clarifying that a large number of regressors would slow down the REML voxel loop - I assumed as much, but wanted to be sure it wasn’t a symptom of a larger problem since my execution time went from 3 hours to 12. Is that portion of the code ran in parallel?

It thought it was an afni_proc error, but I think I may have identified the actual source: tedana’s midk_rejected.txt can overlap with rejected.txt. These text files specify which components are bad, and should be punished by removal. However, they are not exclusive.
rejected.txt:


cat rejected.txt |tr ',' ' ' 
37 46 47 52 57 59 60 61 64 65 66 68 69 70 75 76 77 78 79 80 81 82 83 84 85 87 88 89 90 91 92 93 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 155 156 157

midk_rejected.txt"


cat midk_rejected.txt |tr ',' ' ' 
20 25 26 29 31 33 42 43 50 51 52 54 55 64 73 74d

To save you the trouble of hunting, it looks like #52 and #64 are repeated between lists. I guess that would cause trouble for @extract_meica_ortvec…
For completion sake, regarding cormat warnings:


1d_tool.py -infile meica_mix.1D -show_cormat_warnings

gives


Warnings regarding Correlation Matrix: meica_mix.1D

  severity   correlation   cosine  regressor pair
  --------   -----------   ------  ----------------------------------------
  high:        -0.765      -0.765  #88  vs.  #119
  high:        -0.756      -0.756  #25  vs.  #26
  high:         0.755       0.755  #87  vs.  #119
  high:        -0.743      -0.743  #87  vs.  #88
  high:        -0.719      -0.719  #61  vs.  #85
  high:         0.715       0.715  #61  vs.  #77
  medium:      -0.698      -0.698  #61  vs.  #145
[rest truncated]

However, running:


d_tool.py -show_cormat_warnings -infile meica_orts.1D

yields:


Warnings regarding Correlation Matrix: meica_orts.1D

  severity   correlation   cosine  regressor pair
  --------   -----------   ------  ----------------------------------------
  IDENTICAL:    1.000       1.000  # 3  vs.  #104
  high:         1.000       1.000  # 8  vs.  #107
  high:        -0.705      -0.705  #95  vs.  #96
  medium:      -0.693      -0.693  #26  vs.  #56
  medium:      -0.679      -0.679  #25  vs.  #26
  medium:      -0.671      -0.671  # 7  vs.  #19
  medium:      -0.660      -0.660  # 7  vs.  #24
[remainder truncated]

with those top two being the main problem.

4

Thanks a lot for the details and the sample files.
I have checked in an update to @extract_meica_ortvec
that applies only the unique terms from the component
lists. It is at github now if you would like to give it a try.

Thanks again!

  • rick