QC recommendations after running fMRIPrep + partial afni_proc.py (mask/scale/regress only)

Hi Lauren,
Here is an example of getting Ubuntu 16 binaries, but not trying to overwrite the package that you currently use. Specifying a new -bindir (directory for the binaries) means having to change your PATH to have that new directory in it (at least before the old one).
Try something like this, but feel free to specify the 'bindir' of your choice:

@update.afni.binaries -package linux_ubuntu_16_64.tgz \
                      -bindir ~/abin.ubuntu16

Does that seem reasonable? It basicallly just gets that new package into the directory that you specify. You can move (or delete) the resulting directory (e.g. abin.ubuntu16) as you see fit to.

-rick

Hi Rick - thank you! That worked. Our IT director is also following up on the afni_make_build option to test it out - he said "there is a new afni_make_build as of 4 hours ago. Is that the one you need?" Just wanted to confirm - that is the one to move forward with?
Best,
LL

Hi Lauren,

Yes, the docker file should be current, and compiling locally would work. They will both have your update.

But the @update.afni.binaries command will not yet be sufficient. Our local build process is still running, and it might be an additional couple of hours before the official binaries will be ready for download. So if you run the @uab command now, it will not have the update that is for you. I will write another message once that is ready.

For the record, we do not have a lot of local expertise with Docker. If you use that, converted to aSingularity or Apptainer image, please let us know how it goes.

Thanks,

-rick

Hi Rick - our head of IT at the cluster did the following:

singularity build afni_26.0.02.sif docker://afni/afni_make_build:AFNI_26.0.02

After this, I was able to successfully run afniproc - and it generated the QC files/folder successfully.

Thank you all again for your time and help!
Best,
Lauren

Hi, Lauren-

That is great to know that that worked for creating Singularity.

And also glad that the APQC HTML files can be successfully created. Out of curiosity, is the full set of stats images there?

--pt

Hi Paul - I think so? Here's the directory:

3dClustSim.ACF.cmd
3dREMLfit.err
all_betas.sNTD009_REML+tlrc.BRIK
all_betas.sNTD009_REML+tlrc.HEAD
all_runs.sNTD009+tlrc.BRIK
all_runs.sNTD009+tlrc.HEAD
blur.err_reml.1D
blur_est.sNTD009.1D
corr_brain_blur_for_qc.nii.gz
corr_brain+tlrc.BRIK
corr_brain+tlrc.HEAD
@epi_review.sNTD009
errts.sNTD009_REML_blur_for_qc.nii.gz
errts.sNTD009_REML+tlrc.BRIK
errts.sNTD009_REML+tlrc.HEAD
files_ACF
files_ClustSim
fitts.sNTD009_REML+tlrc.BRIK
fitts.sNTD009_REML+tlrc.HEAD
full_mask.sNTD009+tlrc.BRIK
full_mask.sNTD009+tlrc.HEAD
gtkyd
gtkyd.xls
ideal_Famous_Female_times.1D
ideal_Famous_Male_times.1D
ideal_Other_Female_times.1D
ideal_Other_Male_times.1D
ideal_Self_Female_times.1D
ideal_Self_Male_times.1D
mask_anat.sNTD009+tlrc.BRIK
mask_anat.sNTD009+tlrc.HEAD
mask_epi_anat.sNTD009+tlrc.BRIK
mask_epi_anat.sNTD009+tlrc.HEAD
mean.errts.1D
mean.errts.unit.1D
motion_demean.1D
motion_deriv.1D
motion_sNTD009_enorm.1D
NTD009_motion6_notme_all.1D
out.4095_all.txt
out.4095_warn.txt
out.ap_uvars.json
out.ap_uvars.txt
out.cormat_warn.txt
outcount.r01.1D
outcount.r02.1D
outcount.r03.1D
outcount_rall.1D
out.df_info.txt
out.gcor.1D
out.gtkyd.outliers.txt
out.mask_ae_dice.txt
out.mask_ae_overlap.txt
out.pre_ss_warn.txt
out.ss_review.sNTD009.txt
out.ss_review_uvars.json
out.vlines.pb00.tcat.txt
out.xmat_warn.txt
pb00.sNTD009.r01.tcat+tlrc.BRIK
pb00.sNTD009.r01.tcat+tlrc.HEAD
pb00.sNTD009.r02.tcat+tlrc.BRIK
pb00.sNTD009.r02.tcat+tlrc.HEAD
pb00.sNTD009.r03.tcat+tlrc.BRIK
pb00.sNTD009.r03.tcat+tlrc.HEAD
pb01.sNTD009.r01.scale+tlrc.BRIK
pb01.sNTD009.r01.scale+tlrc.HEAD
pb01.sNTD009.r02.scale+tlrc.BRIK
pb01.sNTD009.r02.scale+tlrc.HEAD
pb01.sNTD009.r03.scale+tlrc.BRIK
pb01.sNTD009.r03.scale+tlrc.HEAD
QC_sNTD009
radcor.pb02.regress
run_graphview_errts.tcsh
run_graphview_pbrun.tcsh
run_instacorr_errts.tcsh
run_instacorr_pbrun.tcsh
run_qc_00_vorig_EPI-tcat.tcsh
run_qc_01_vorig_EPI_variance.tcsh
run_qc_02_vorig_anat.tcsh
run_qc_03_vstat_Full_Fstat.tcsh
run_qc_04_vstat_Famous_Female_times_0_Coef.tcsh
run_qc_05_vstat_Famous_Male_times_0_Coef.tcsh
run_qc_06_vstat_Other_Female_times_0_Coef.tcsh
run_qc_07_vstat_Other_Male_times_0_Coef.tcsh
run_qc_14_regr_corr_errts.tcsh
run_qc_15_radcor_rc_regress_d02.tcsh
@ss_review_basic
@ss_review_driver
@ss_review_driver_commands
stats.REML_cmd
stats.sNTD009_REML+tlrc.BRIK
stats.sNTD009_REML+tlrc.HEAD
stats.sNTD009_REMLvar+tlrc.BRIK
stats.sNTD009_REMLvar+tlrc.HEAD
stimuli
sub-NTD009_ses-2_space-MNI152NLin6Asym_res-2_desc-preproc_T1w_brain+tlrc.BRIK
sub-NTD009_ses-2_space-MNI152NLin6Asym_res-2_desc-preproc_T1w_brain+tlrc.HEAD
sum_ideal.1D
TSNR.sNTD009+tlrc.BRIK
TSNR.sNTD009+tlrc.HEAD
tsnr_stats_regress
vlines.pb00.tcat
X.jpg
X.stim.xmat.1D
X.xmat.1D

Best,
LL

Hi, Lauren-

Thanks for that list, and I do see the stats* datasets there.

I was also wondering if the APQC HTML includes the usual montage images of the statistics volumes for quick viewing, under the "vstat" section in the APQC HTML.

If you run:

open_apqc.py -infiles QC_sNTD009/index.html

... and then click on the "vstat" tab in the upper part of the webpage that opens, are there images of the full F-stat of the modeling, as well as main effects (and possibly contrasts, though by default only 5 stats volumes are shown? NB: more can be, using options in AP).

--pt

That's great, Lauren, and thanks for sharing the singularity command!

-rick

Hi Paul - yes just tried that command and the images of the full F-stat and main effects are there!

Hi, Lauren-

That's great to know, thanks.

By default, it will put 5 the first 5 items from among the following, in descending order: full F-stat, main effects, contrasts modeled with 3dDeconvolve. So, if you had 3 stimulus classes and 3 contrasts, you would only see the Full F-stat (always present), the main effects of each stimulus class, and 1 contrasts.

If you wanted to show more (or less) in that section, you can add an option to AP to explicitly list which ones to see, via their labels:

-html_review_opts -vstat_list label1 [label2 label3 ...]

Note that the full F-stat need not be included there, it will always be shown.

Alternatively, if you have already run AP and don't want to re-run it, you could regenerate the APQC HTML with more vstats by running the following in the AP results directory:

apqc_make_tcsh.py                        \
   -uvar_json   out.ss_review_uvars.json \
   -subj_dir    .                        \
   -vstat_list  label1 [label2 label3 ...]

Re. the labels themselves, those are simply the same ones used in the AP command to define the stimulus classes and the GLT formula. From looking at the stats*HEAD file subbrick labels, you can also extract them as follows (from the apqc_make_tcsh.py help):

-vstat_list A B C ...
                  :(opt, only applicable if stim timing is used in
                   processing) provide a list of label items to specify
                   which volumes's images should appear in the vstat
                   QC block.  Each item should correspond to subbrick
                   label basename (so not including '_GLT', "#N",
                   'Tstat', 'Fstat', 'Coef', etc.) in the stats_dset.
                   'Full_Fstat' is always added/included, even if not
                   provided in this list. If not used, the program
                   uses default logic to pick up to 5 items to show.

Just in case that is helpful.

--pt

Hi Rick & Paul,

I sent an email about this so I could include our IT Director, but following up here too to flag it - while the afniproc script ran successfully after using the docker, we've run into an error with the Docker setup using 3dMVM:

When I try to run a 3dMVM script, it crashes with the following error:

Error in library(data.table) : there is no package called ‘data.table’

We suspect this indicates the Docker image may not have been created correctly. Any guidance on how to resolve this would be greatly appreciated.

Thank you for your time!
Lauren

Hi, Lauren-

I believe that the issue here is likely the R version, which from the afni_system_check.py output above is:

R version            : R version 3.4.4 (x86_64-pc-linux-gnu)

That is pretty old (maybe from 2018?).

We have generally been pointing at R version 4.4.* in the install instructions. (The current release is 4.5.*, but at least on macOS, R v4.5.1 had some sub-library problems, so that is another reason why 4.4.* is a good version to consider; we haven't yet been able to test the very latest R v4.5.2 to see if that release has included fixes for those sub-packages.)

So, is it possible for R to be updated there?

--pt

Hi,

Our latest unreleased version of the afni docker image is here.

Your IT director can pull that image directly.
Or you can build it from source from our github here.

There is a file "temp_notes.txt" in the github repository linked above that has instructions for building or running that version of the docker image. The instructions in the text file should work with most flavors of linux but not macOS.
The direct link is here.

Thanks, Justin

1 Like

Thanks Justin! Two quick follow-ups:

  • Does this docker versions include the fix from January 13 (see above)?

  • And does this docker version now include the R packages as well?

Thanks for clarifying!
Best,
LL

Hi Justin - it appears the docker PATH is not set to have /usr/local/abin in it. Could this be fixed?
Thank you!
Best,
LL

Hi,

I pushed a quick fix on docker hub with the tags 1.7 or latest.
It has the latest public version of the afni binaries.
It also has a copy of afni_user's .bashrc in the /root folder so you can run with "--user root" if you need to for singularity.
That adds /usr/local/abin to the path for root.
I have run some basic testing with it, but I have not thoroughly tested it.

Let us know how it goes.

Thanks, Justin