Using Tedana via AFNI_proc.py

Hi all-

I’m trying to use the tedana functionality built in to AFNI_proc.py. This is my first attempt with multi-echo data and I’m running into a problem. Here is the AFNI_proc.py script for some rest data:


afni_proc.py \
-subj_id ${subject} \
-script proc.${subject}_MBME_rest \
-scr_overwrite \
-out_dir ${resdir}/${subject}_results_MBME_rest \
-dsets_me_run ${subdir}/Nifti/${subject}/ses-001/func/${subject}_ses-001_restMBME_run-001_echo-*_bold.nii.gz \
-echo_times 12.8 30.4 48 \
-reg_echo 2 \
-copy_anat ${subdir}/results/${subject}/fs/${subject}/SUMA/T1.nii \
-anat_has_skull yes \
-anat_follower_ROI FSvent epi ${resdir}/masks/${subject}_vent_mask.nii \
-anat_follower_ROI FSwm epi ${resdir}/masks/${subject}_wm_mask.nii \
-anat_follower_erode FSvent FSwm \
-blocks despike tshift align tlrc volreg blur mask combine scale regress \
-blip_forward_dset ${subdir}/Nifti/${subject}/ses-001/fmap/${subject}_ses-001_fmap_run-001_phasediff.nii.gz \
-blip_reverse_dset ${subdir}/Nifti/${subject}/ses-001/fmap/${subject}_ses-001_fmap_run-002_phasediff.nii.gz \
-despike_new yes \
-tlrc_base MNI152_T1_2009c+tlrc \
-tlrc_NL_warp \
-align_opts_aea \
-giant_move \
-cost lpc+ZZ \
-volreg_align_to MIN_OUTLIER \
-volreg_tlrc_warp \
-volreg_align_e2a \
-combine_method OC_tedort \
-blur_in_automask \
-mask_apply anat \
-regress_motion_per_run \
-regress_ROI_PC FSvent 3 \
-regress_ROI_PC_per_run FSvent \
-regress_anaticor_fast \
-regress_anaticor_label FSwm \
-test_stim_files no \
-regress_opts_3dD \
-jobs 4 \
-regress_bandpass 0.01 0.1 \
-regress_censor_motion 0.2 \
-regress_apply_mot_types demean deriv \
-regress_censor_first_trs 4 \
-regress_est_blur_errts \
-remove_preproc_files \
-execute

end


I get the following text output to the terminal:


3dcalc -float  -a /projects/MBME_test/results/SKY02400_results_MBME_rest/tedana_r01/tedprep_echo_3_masked+tlrc -expr 'a*10000/5398.880684' -prefix /projects/MBME_test/results/SKY02400_results_MBME_rest/tedana_r01/tedprep_echo_3_preZcat
Output Zcat dataset is /projects/MBME_test/results/SKY02400_results_MBME_rest/tedana_r01/tedprep_Zcat_all.nii.gz


python /home/doug/abin/meica.libs/tedana.py \
--label r01 \
--orig_data /projects/MBME_test/results/SKY02400_results_MBME_rest/tedana_r01/tedprep_Zcat_all.nii.gz \
--TEs 12.8,30.4,48.0 \


  File "/home/doug/abin/meica.libs/tedana.py", line 77
    print '-- initializing random seed to %s' % seed
          ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print('-- initializing random seed to %s' % seed)?
end
mkdir meica_orts
foreach run ( 01 )
@extract_meica_ortvec -meica_dir tedana_r01/TED.r01 -reject_midk 1 -work_dir tedana_r01/work.orts -prefix tedana_r01/meica_orts.1D
-- computing projection terms from MEICA results
** missing input meica parameter file, tedana_r01/TED.r01/accepted.txt


The Zcat file exists, but the TED.r01 directory has not been created. I thought maybe this was a python version issue so I ran the -find_prog python flag on system check and get the following output:


/home/doug/freesurfer/bin/fspython
   /usr/bin/python3.8
   /usr/bin/python3-futurize
   /usr/bin/python3-pasteurize
   /home/doug/abin/afni_python_wrapper.py
   /home/doug/abin/python_module_test.py


proc.py seems to be calling the tedana.py file from the abin directory, so maybe it isn’t a python version problem?

Here are the results of my system check if that is useful:


-------------------------------- general ---------------------------------
architecture:         64bit ELF
system:               Linux
release:              5.13.0-44-generic
version:              #49~20.04.1-Ubuntu SMP Wed May 18 18:44:28 UTC 2022
distribution:         ubuntu 20.04 focal
number of CPUs:       8
apparent login shell: bash
shell RC file:        .bashrc (exists)

--------------------- AFNI and related program tests ---------------------
which afni           : /home/doug/abin/afni
afni version         : Precompiled binary linux_ubuntu_16_64: Jun  2 2022 
                     : AFNI_22.1.12 'Antoninus Pius'
AFNI_version.txt     : AFNI_22.1.12, linux_ubuntu_16_64, Jun 02 2022
which python         : /usr/bin/python
python version       : 3.8.10
which R              : /usr/bin/R
R version            : R version 4.2.0 (2022-04-22) -- "Vigorous Calisthenics"
which tcsh           : /usr/bin/tcsh

instances of various programs found in PATH:
    afni    : 1   (/home/doug/abin/afni)
    R       : 1   (/usr/bin/R)
    python  : 1   (/usr/bin/python3.8)
    python2 : 0 
    python3 : 1   (/usr/bin/python3.8)

** have python3 but not python2

testing ability to start various programs...
    afni                 : success
    suma                 : success
    3dSkullStrip         : success
    uber_subject.py      : success
    3dAllineate          : success
    3dRSFC               : success
    SurfMesh             : success
    3dClustSim           : success
    3dMVM                : success

checking for R packages...
    rPkgsInstall -pkgs ALL -check : success

R RHOME : /usr/lib/R

checking for $HOME files...
    .afnirc                   : found
    .sumarc                   : found
    .afni/help/all_progs.COMP : found

------------------------------ python libs -------------------------------
** failed to load module PyQt4
-- PyQt4 is no longer needed for an AFNI bootcamp

++ module loaded: matplotlib.pyplot
   module file : /usr/lib/python3/dist-packages/matplotlib/pyplot.py
   matplotlib version : 3.1.2

-------------------------------- env vars --------------------------------
PATH = /home/doug/freesurfer/bin:/home/doug/freesurfer/fsfast/bin:/home/doug/freesurfer/tktools:/home/doug/freesurfer/mni/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/doug/abin:/usr/local/MATLAB/R2022a/bin

PYTHONPATH = 
R_LIBS = /home/doug/R

LD_LIBRARY_PATH = 
DYLD_LIBRARY_PATH = 
DYLD_FALLBACK_LIBRARY_PATH = 

------------------------------ data checks -------------------------------
data dir : found AFNI_data6   under $HOME (110653M Avail)
           top history: 20 Feb 2020 [rickr]: updated FT_analysis examples
data dir : found AFNI_demos   under $HOME
           top history: ...ct 2020 [taylorp]: updated scripts under FATCAT_DEMO
data dir : found suma_demo    under $HOME
           top history: ...s_New/data/Build_tmp on Mon Mar  4 11:56:45 EST 2013
data dir : found afni_handouts under $HOME
atlas    : found TT_N27+tlrc  under /home/doug/abin

------------------------------ OS specific -------------------------------
which apt-get        : /usr/bin/apt-get
apt-get version      : apt 2.0.6 (amd64)

have Ubuntu system: ubuntu 20.04 focal
have Ubuntu afni  : Precompiled binary linux_ubuntu_16_64: Jun  2 2022 

=========================  summary, please fix:  =========================
*  just be aware: login shell 'bash', but our code examples use 'tcsh'


Is my problem as simple as setting the python path in AFNI?

I’d be happy to upload the full text output if that would be helpful.

Any advice would be appreciated. I’m not sure what troubleshooting steps to try next.

Thanks,
Doug

Hi Doug,
Which tedana version would you like to use? If you want to use the tedana packaged in abin/, you’ll probably need to use a python 2 environment as it’s python 2. In order to use a more modern tedana (which would be my current recommendation), you’ll want to, in your current environment, run:

pip install tedana

to get the latest version. Then, in your afni_proc script, change the line
-combine_method OC_tedort \

to

-combine_method m_tedana_m_tedort \

Relevant excerpt of the afni_proc help text:

---- combine methods that use Prantik’s “original” tedana.py ----

           Prantik's tedana.py is run using the 'tedana*' combine methods.

              Prantik's tedana.py requires python 2.7.

              By default, tedana.py will be applied from the AFNI
              installation directory.

              Alternatively, one can specify the location of a different
              tedana.py using -combine_tedana_path.  And if it is 
              preferable to run it as an executable (as opposed to running
              it via 'python PATH/TO/tedana.py'), one can tell this to
              tedana_wrapper.py by applying:
                     -combine_opts_tedwrap -tedana_is_exec

            methods
            -------
            OC_tedort        : OC, and pass tedana orts to regression
            tedana           : run tedana.py, using output dn_ts_OC.nii
            tedana_OC        : run tedana.py, using output ts_OC.nii
                               (i.e. use tedana.py for optimally combined)
            tedana_OC_tedort : tedana_OC, and include tedana orts


        ---- combine methods that use tedana from the MEICA group ----

           The MEICA group tedana is specified with 'm_tedana*' methods.

              This tedana requires python 3.6+.

              AFNI does not distribute this version of tedana, so it must
              be in the PATH.  For installation details, please see:

                 https://tedana.readthedocs.io/en/stable/installation.html

            methods
            -------
            m_tedana         : tedana from MEICA group (dn_ts_OC.nii.gz)
            m_tedana_OC      : tedana OC from MEICA group (ts_OC.nii.gz)
            m_tedana_m_tedort: tedana from MEICA group (dn_ts_OC.nii.gz)
                               "tedort" from MEICA group
                               (--tedort: "good" projected from "bad")

Please feel free to ask me more questions as I’m one of the tedana developers.

Indeed. As Josh suggested, if you are new to tedana, start with the one from the MEICA group (which does not come with AFNI).

Also, your blip_forward and blip_reverse datasets do not look like forward and reverse phase encoding EPI, but rather actual field maps. Computation of the correction map would need to be done separately, possibly via epi_b0_correct.py. And to apply there, there is an update that I have still not gotten around to doing in afni_proc.py, which someone has been waiting for a while on, unfortunately…

  • rick

Thank-you. I got tedana up and running with no errors.

Hi Rick,

Thanks for the quick response. I haven’t done much with distortion correction. I think the issue might be with how I named files. The inputs don’t look like field maps to me, I’m not sure how BIDS would categorize them, might need to do some more research on that.

Here is the content of the json from one of the scans I’m using as blip_forward:


{
  "AcquisitionMatrixPE": 84,
  "AcquisitionNumber": 1,
  "AcquisitionTime": "12:43:56.152500",
  "BandwidthPerPixelPhaseEncode": 40.355,
  "BaseResolution": 84,
  "BodyPartExamined": "BRAIN",
  "ConsistencyInfo": "N4_VE11C_LATEST_20160120",
  "ConversionSoftware": "dcm2niix",
  "ConversionSoftwareVersion": "v1.0.20190410  GCC6.3.0",
  "DerivedVendorReportedEchoSpacing": 0.000590002,
  "DeviceSerialNumber": "45487",
  "DwellTime": 2.9e-06,
  "EchoTime": 0.0298,
  "EchoTrainLength": 41,
  "EffectiveEchoSpacing": 0.000295001,
  "FlipAngle": 60,
  "ImageComments": "Unaliased_MB3_PE4_LB",
  "ImageOrientationPatientDICOM": [
    0.998303,
    0.0259533,
    0.0521248,
    -0.0191684,
    0.991758,
    -0.126686
  ],
  "ImageType": [
    "ORIGINAL",
    "PRIMARY",
    "M",
    "MB",
    "ND",
    "MOSAIC"
  ],
  "ImagingFrequency": 123.204,
  "InPlanePhaseEncodingDirectionDICOM": "COL",
  "InstitutionAddress": "L_W_Chase_Hall_203_Lincoln_District_US_68583",
  "InstitutionName": "UNL_CB3",
  "InstitutionalDepartmentName": "Department",
  "MRAcquisitionType": "2D",
  "MagneticFieldStrength": 3,
  "Manufacturer": "Siemens",
  "ManufacturersModelName": "Skyra",
  "Modality": "MR",
  "MultibandAccelerationFactor": 3,
  "ParallelReductionFactorInPlane": 2,
  "PartialFourier": 1,
  "PatientPosition": "HFS",
  "PercentPhaseFOV": 100,
  "PhaseEncodingDirection": "j-",
  "PhaseEncodingSteps": 83,
  "PhaseResolution": 1,
  "PixelBandwidth": 2055,
  "ProcedureStepDescription": "CB3_Schultz",
  "ProtocolName": "cmrr_mbep2d_bold_MB3_REST",
  "PulseSequenceDetails": "%CustomerSeq%_cmrr_mbep2d_bold",
  "ReceiveCoilActiveElements": "HEA;HEP",
  "ReceiveCoilName": "Head_32",
  "ReconMatrixPE": 84,
  "RefLinesPE": 24,
  "RepetitionTime": 1,
  "SAR": 0.141214,
  "ScanOptions": "FS",
  "ScanningSequence": "EP",
  "SequenceName": "epfid2d1_84",
  "SequenceVariant": "SK_SS",
  "SeriesDescription": "cmrr_mbep2d_bold_MB3_REST",
  "SeriesNumber": 10,
  "ShimSetting": [
    4176,
    -10609,
    11812,
    305,
    761,
    -65,
    139,
    211
  ],
  "SliceThickness": 2.5,
  "SliceTiming": [
    0,
    0.5175,
    0.0575,
    0.575,
    0.115,
    0.6325,
    0.1725,
    0.69,
    0.23,
    0.7475,
    0.2875,
    0.8075,
    0.345,
    0.865,
    0.4025,
    0.9225,
    0.46,
    0,
    0.5175,
    0.0575,
    0.575,
    0.115,
    0.6325,
    0.1725,
    0.69,
    0.23,
    0.7475,
    0.2875,
    0.8075,
    0.345,
    0.865,
    0.4025,
    0.9225,
    0.46,
    0,
    0.5175,
    0.0575,
    0.575,
    0.115,
    0.6325,
    0.1725,
    0.69,
    0.23,
    0.7475,
    0.2875,
    0.8075,
    0.345,
    0.865,
    0.4025,
    0.9225,
    0.46
  ],
  "SoftwareVersions": "syngo_MR_E11",
  "SpacingBetweenSlices": 2.5,
  "StationName": "AWP45487",
  "TotalReadoutTime": 0.0244851,
  "TxRefAmp": 355.89,
  "WipMemBlock": "3ff044d6-152d-405f-a10b-4a8da4ed1e5b||Sequence: R016 ve11c/master r/8780670; Nov 27 2017 15:05:12 by eja"
}

The other file has the same parameters with the exception of PhaseEncodingDirection which is “j”. Are these the correct inputs for blip_forward and blip_reverse, or should I be using a different approach? I haven’t implemented distortion correction previously. Honestly, my data without it looks pretty good, but I’m curious to see if it makes a difference.

That json file has PhaseEncodingDirection of j-, it would be nice to verify whether the other json file has j+ for that.

The reason I say anything is because the files are named as fmap_run-001_phasediff…, for example. phasediff seems a little odd for a standard EPI acquisition (just possibly in the reverse direction). Also, the files are named as run-001 and run-002. They should look just like the other EPI volumes.

  • rick

Hi Rick,

The other file is definitely the opposite direction (j+).

In the future, would it make more sense to collect a short EPI with the same parameters as my other scans with the opposite phase? If so, is there an optimal duration or number of TRs for using blip_reverse?

Thanks,
Doug

Hi Doug,

Yes, the reverse phase encoding scan should be identical to the forward one, except for that direction.
Typically, one might collect 10 reverse encoding volumes before starting a normal scan with forward encoding. Then the first 10 (or whatever matching number) of the full forward run can be extracted and used for the distortion correction. No separate forward scan is needed, since the task/rest scan volume will work just as well. Sometimes people use only 1 volume, but make it the first pre-steady state one (with better contrast for registration). It might be safer to collect the “full” 10 though, just in case something bad happens early.

It is preferable to collect the reverse just before the forward, to minimize subject motion between the sets of volumes.

If you do not explicitly give afni_proc.py a forward set, it will use the first N volumes from the first run (where N = number of reverse volumes).
It is preferable to provide it explicitly, to be sure (e.g. if reverse wasn’t before the first run).

  • rick