obliquity_remover.py questions

Hi Paul,

I started to play with obliquity_remover.py, and It sounds useful to apply this to EPI data, as it preserves the original correspondence with the T1w image.

May I ask two questions:

  1. I assume that, if I use the blip up/down method for distortion correction, it is better to treat the AP and PA EPI datasets also as -child_dsets, right?

  2. Is it possible to revert a deobliqued dataset back to the original “raw” version? I’m asking because saving the output with _DEOB essentially duplicates the dataset after a simple modification. Deobliquing seems preferable, so it may not be worth keeping a separate copy of the raw data. If it can be reverted when needed, that would be ideal, just in case someone needs the original for a specific reason.

Thanks,
Zhengchen

I tested on an old dataset.

3dinfo -is_oblique func/*_run-1_bold_DEOB.nii.gz return me 1, is this expected? Please find the detailed code below

> 3dinfo -is_oblique anat/sub-ep0568_T1w.nii.gz
1

> 3dinfo -is_oblique func/sub-ep0568_task-spike_run-1_bold.nii.gz
1

> obliquity_remover.py -inset anat/sub-ep0568_T1w.nii.gz -prefix anat/sub-ep0568_T1w_DEOB.nii.gz -child_dsets func/sub-ep0568_task-spike_run-1_bold.nii.gz -child_prefixes func/sub-ep0568_task-spike_run-1_bold_DEOB.nii.gz -overwrite -do_qc 0
++ Removing obliquity from header via mode: keep_origin_raw
++ Process child dset: func/sub-ep0568_task-spike_run-1_bold.nii.gz
++ Applying obliquity

> 3dinfo -is_oblique anat/sub-ep0568_T1w_DEOB.nii.gz
0

> 3dinfo -is_oblique func/sub-ep0568_task-spike_run-1_bold_DEOB.nii.gz
1

> 3dinfo anat/sub-ep0568_T1w_DEOB.nii.gz
++ 3dinfo: AFNI version=AFNI_25.3.04 (Dec 24 2025) [64-bit]

Dataset File:    /Volumes/django/test_deoblique/sub-ep0568/anat/sub-ep0568_T1w_DEOB.nii.gz
Identifier Code: AFN_YbGukYJDp28zCfngVSNOvA  Creation Date: Sat Jan  3 14:00:53 2026
Template Space:  ORIG
Dataset Type:    Anat Bucket (-abuc)
Byte Order:      LSB_FIRST {assumed} [this CPU native = LSB_FIRST]
Storage Mode:    NIFTI
Storage Space:   23,068,672 (23 million) bytes
Geometry String: "MATRIX(-0.999998,0,0,78.13178,0,-1,0,80.25916,0,0,1,-141.9489):176,256,256"
Data Axes Tilt:  Plumb
Data Axes Orientation:
  first  (x) = Left-to-Right
  second (y) = Posterior-to-Anterior
  third  (z) = Inferior-to-Superior   [-orient LPI]
R-to-L extent:   -96.868 [R] -to-    78.132 [L] -step-     1.000 mm [176 voxels]
A-to-P extent:  -174.741 [A] -to-    80.259 [P] -step-     1.000 mm [256 voxels]
I-to-S extent:  -141.949 [I] -to-   113.051 [S] -step-     1.000 mm [256 voxels]
Number of values stored at each pixel = 1
  -- At sub-brick #0 '#0' datum type is short:            0 to           559

----- HISTORY -----
[django@django: Sat Jan  3 14:00:52 2026] {AFNI_25.3.04:macos_13_ARM} 3dcopy anat/sub-ep0568_T1w.nii.gz anat/__workdir_adfo_JKvjEdhT6pW/tmp
[django@django: Sat Jan  3 14:00:53 2026] {AFNI_25.3.04:macos_13_ARM} 3drefit -oblique_recenter_raw tmp+orig
[django@django: Sat Jan  3 14:00:53 2026] {AFNI_25.3.04:macos_13_ARM} 3drefit -deoblique tmp+orig
[django@django: Sat Jan  3 14:00:53 2026] {AFNI_25.3.04:macos_13_ARM} 3dcopy tmp+orig ../sub-ep0568_T1w_DEOB.nii.gz
[django@django: Sat Jan  3 14:00:54 2026] /opt/abin/obliquity_remover.py -inset anat/sub-ep0568_T1w.nii.gz -prefix anat/sub-ep0568_T1w_DEOB.nii.gz -child_dsets func/sub-ep0568_task-spike_run-1_bold.nii.gz -child_prefixes func/sub-ep0568_task-spike_run-1_bold_DEOB.nii.gz -overwrite -do_qc 0

> 3dinfo func/sub-ep0568_task-spike_run-1_bold_DEOB.nii.gz
++ 3dinfo: AFNI version=AFNI_25.3.04 (Dec 24 2025) [64-bit]

Dataset File:    /Volumes/django/test_deoblique/sub-ep0568/func/sub-ep0568_task-spike_run-1_bold_DEOB.nii.gz
Identifier Code: AFN_lNCdBrqDlSInGCBdJ7KZbg  Creation Date: Sat Jan  3 14:00:56 2026
Template Space:  ORIG
Dataset Type:    Echo Planar (-epan)
Byte Order:      LSB_FIRST {assumed} [this CPU native = LSB_FIRST]
Storage Mode:    NIFTI
Storage Space:   40,960,000 (41 million) bytes
Geometry String: "MATRIX(4.99888,-0.104503,0.016589,-164.218,-0.103244,-4.9889,-0.316526,134.289,-0.023167,-0.316112,4.98994,-19.2231):64,64,25"
Data Axes Tilt:  Oblique (3.818 deg. from plumb)
Data Axes Approximate Orientation:
  first  (x) = Right-to-Left
  second (y) = Posterior-to-Anterior
  third  (z) = Inferior-to-Superior   [-orient RPI]
R-to-L extent:  -164.218 [R] -to-   150.782 [L] -step-     5.000 mm [ 64 voxels]
A-to-P extent:  -180.711 [A] -to-   134.289 [P] -step-     5.000 mm [ 64 voxels]
I-to-S extent:   -19.223 [I] -to-   100.777 [S] -step-     5.000 mm [ 25 voxels]
Number of time steps = 200  Time step = 1.75000s  Origin = 0.00000s  Number time-offset slices = 25  Thickness = 5.000
  -- At sub-brick #0 'sub-ep0568_task-[0]' datum type is short:            0 to           833
  -- At sub-brick #1 'sub-ep0568_task-[1]' datum type is short:            0 to           613
  -- At sub-brick #2 'sub-ep0568_task-[2]' datum type is short:            0 to           591
** For info on all 200 sub-bricks, use '3dinfo -verb' **

----- HISTORY -----
[django@django: Sat Jan  3 14:00:55 2026] {AFNI_25.3.04:macos_13_ARM} 3dTcat -prefix func/__wdir_child_sub-ep0568_task-spike_run-1_bold_DEOB_NsB8qj2je1k/child_00 func/sub-ep0568_task-spike_run-1_bold.nii.gz
[django@django: Sat Jan  3 14:00:55 2026] {AFNI_25.3.04:macos_13_ARM} 3drefit -atrfloat IJK_TO_DICOM_REAL child_01_aform.aff12.1D child_00+orig.HEAD
[django@django: Sat Jan  3 14:00:56 2026] {AFNI_25.3.04:macos_13_ARM} 3drefit -atrfloat IJK_TO_DICOM child_01_ijk2dicom.aff12.1D child_00+orig.HEAD
[django@django: Sat Jan  3 14:00:56 2026] {AFNI_25.3.04:macos_13_ARM} 3drefit -atrfloat ORIGIN child_01_origin.1D child_00+orig.HEAD
[django@django: Sat Jan  3 14:00:56 2026] {AFNI_25.3.04:macos_13_ARM} 3dcopy child_00+orig.HEAD ../sub-ep0568_task-spike_run-1_bold_DEOB.nii.gz

QC seems not working, it took a while (7-min for one bold run) and returned warning, in the tmp folder I only have img_olap_00_DEOB.txt

> obliquity_remover.py -inset anat/sub-ep0568_T1w.nii.gz -prefix anat/sub-ep0568_T1w_DEOB.nii.gz -child_dsets func/sub-ep0568_task-spike_run-1_bold.nii.gz -child_prefixes func/sub-ep0568_task-spike_run-1_bold_DEOB.nii.gz -overwrite -do_qc 1
++ Removing obliquity from header via mode: keep_origin_raw
++ Process child dset: func/sub-ep0568_task-spike_run-1_bold.nii.gz
++ Make QC image, child olap 'in'
+* WARNING: Cannot find any QC image: func/__wdir_child_sub-ep0568_task-spike_run-1_bold_DEOB_o81pCvZef37/img_olap_00*
++ Applying obliquity
++ Make QC image, child olap 'out'
+* WARNING: Cannot find any QC image: func/__wdir_child_sub-ep0568_task-spike_run-1_bold_DEOB_o81pCvZef37/img_olap_01*

Hi, @zhengchencai:

I'm running out the door, but could I just ask what your AFNI version is (afni -ver), re. the speed issue?

--pt

Hi @ptaylor, no rush at all, just want to provide some test feedback.

> afni -ver
Precompiled binary macos_13_ARM: Dec 24 2025 (Version AFNI_25.3.04 'Gordian II')

Here is my afni_system_check.py -check_all output

-------------------------------- general ---------------------------------
architecture:         64bit Mach-O
cpu type:             arm64
system:               Darwin
release:              24.6.0
version:              Darwin Kernel Version 24.6.0: Wed Nov  5 21:30:44 PST 2025; root:xnu-11417.140.69.705.2~1/RELEASE_ARM64_T6041
distribution:         15.7.3
number of CPUs:       12
user:                 django
apparent login shell: zsh
shell RC file:        .zshrc (exists)

--------------------- AFNI and related program tests ---------------------
which afni           : /opt/abin/afni
afni version         : Precompiled binary macos_13_ARM: Dec 24 2025
                     : AFNI_25.3.04 'Gordian II'
AFNI_version.txt     : AFNI_25.3.04, macos_13_ARM, Dec 24 2025, build
afnipy version       : AFNI_25.3.04
which python         : /Users/django/miniconda3/bin/python
python version       : 3.13.5
which R              : /usr/local/bin/R
R version            : R version 4.4.1 (aarch64-apple-darwin20)

instances of various programs found in PATH:
    afni    : 1   (/opt/abin/afni)
    R       : 1   (/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/bin/R)
    python  : 2
      /Users/django/miniconda3/bin/python3.13
      /opt/homebrew/Cellar/python@3.13/3.13.5/Frameworks/Python.framework/Versions/3.13/bin/python3.13
    python2 : 0
    python3 : 3
      /Users/django/miniconda3/bin/python3.13
      /opt/homebrew/Cellar/python@3.13/3.13.5/Frameworks/Python.framework/Versions/3.13/bin/python3.13
      /usr/bin/python3

testing ability to start various programs...
    afni                 : success
    suma                 : success
    3dSkullStrip         : success
    3dAllineate          : success
    3dRSFC               : success
    SurfMesh             : success
    3dClustSim           : success
    build_afni.py        : success
    uber_subject.py      : success
    3dMVM                : success
    rPkgsInstall         : success

------------------------ dependent program tests -------------------------
checking for dependent programs...

which tcsh           : /bin/tcsh
tcsh version         : 6.21.00
which Xvfb           : /opt/X11/bin/Xvfb

checking for R packages...
    rPkgsInstall -pkgs ALL -check : success

R RHOME : /Library/Frameworks/R.framework/Resources

------------------------------ python libs -------------------------------

++ module loaded: matplotlib.pyplot
   module file : /Users/django/miniconda3/lib/python3.13/site-packages/matplotlib/pyplot.py
   matplotlib version : 3.10.3

++ module loaded: flask
   module file : /Users/django/miniconda3/lib/python3.13/site-packages/flask/__init__.py
   flask version : 3.1.1

++ module loaded: flask_cors
   module file : /Users/django/miniconda3/lib/python3.13/site-packages/flask_cors/__init__.py
   flask_cors version : 6.0.1

-------------------------------- env vars --------------------------------
PATH                       = /Users/django/.local/bin:/Users/django/.antigravity/antigravity/bin:/opt/X11/bin:/Users/django/.nvm/versions/node/v22.17.0/bin:/Users/django/miniconda3/bin:/Users/django/miniconda3/condabin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Library/Apple/usr/bin:/Applications/quarto/bin:/Applications/iTerm.app/Contents/Resources/utilities:/opt/homebrew/opt/python/libexec/bin:/opt/abin:/opt/ants/bin:/Users/django/.nvm/versions/node/v22.17.0/bin:/opt/homebrew/opt/fzf/bin

PYTHONPATH                 =
R_LIBS                     = /Users/django/sw/R-4.3.1
LD_LIBRARY_PATH            =
DYLD_LIBRARY_PATH (sub-shell) = :/opt/X11/lib/flat_namespace
DYLD_FALLBACK_LIBRARY_PATH (sub-shell) =
CONDA_SHLVL                = 1
CONDA_DEFAULT_ENV          = base
CC                         =
HOMEBREW_PREFIX            = /opt/homebrew

----------------------------- eval dot files -----------------------------

----------- AFNI $HOME files -----------

    .afnirc                   : found
    .sumarc                   : found
    .afni/help/all_progs.COMP : found

--------- shell startup files ----------

   -- good: .tcshrc seems to contain 'source .cshrc'
   -- considered operations: path, apsearch

   -- note: followers should not need edits, so edit flags should be 0
      (have 1 follower(s), which can be ignored)

   no modifications needed across 3 dot files

------------------------------ data checks -------------------------------
data dir : missing AFNI_data6
data dir : missing AFNI_data7
data dir : missing AFNI_demos
data dir : missing suma_demo
data dir : found afni_handouts under $HOME/Postdoc/Tutorial (471G Avail)
atlas    : found TT_N27+tlrc  under /opt/abin

------------------------------ OS specific -------------------------------
XQuartz version      : 2.8.5

which brew           : /opt/homebrew/bin/brew
brew version         : Homebrew 5.0.7

which git            : /opt/homebrew/bin/git
git version          : git version 2.52.0
which gcc            : /usr/bin/gcc
gcc version          : Apple clang version 17.0.0 (clang-1700.6.3.2)

brew gcc(s)          : /opt/homebrew/bin/gcc-15
CommandLineTools SDK : MacOSX26.2.sdk

=========================  summary, please fix:  =========================
*  insufficient data for AFNI bootcamp
   (see "Prepare for Bootcamp" on install pages)

Hi, @zhengchencai :

Thanks for the version info. That is uptodate; some changes had been made prior to that to speed it up for the QC images.

To Q1 above: for the majority of volumetric FMRI processing, users want the EPI data to end up either in participant's own anatomical dataset space (like, the reference T1w anatomical) or in a template. (A differing case not discussed here is where the input EPI grid is desired to be the final space+grid for the FMRI data, like for some high-res acquisitions; we leave that for a different thread.) When wanting to use the participant anatomical or template to define a final space, then what we would like to do when the anatomical has obliquity is:

  • remove the obliquity from the anatomical without blurring it;
  • retain overlap with the EPI, without blurring the EPI either, which can be done by moving over the removed obliquity to the EPI

This is what obliquity_remover.py will do; furthermore, "EPI" can refer to multiple EPI runs simultaneously, whether the main FMRI dsets or the blip up/down partners (or phase datasets) that are on the same grids as the main EPI.

So, the answer to your Q1 is: yes.

Q2: The obliquity_remover.py program also saves out the oblique affine transformation piece that was removed from the anatomical, and saves it to a separate file to be able to apply it again later, should the need arise. Details below.


Details of applying affine after obliquity_remover.py

Setup: Let's say your input anatomical that has obliquity is called AAA.nii.gz.

To remove the obliquity (ignoring child dsets for a moment), you could create dataset BBB.nii.gz (which is not regridded, and shares the same coordinate origin as AAA.nii.gz):

obliquity_remover.py -prefix BBB.nii.gz -inset AAA.nii.gz

... which also creates two files containing the removed obliquity itself, stored as affine matrices:
BBB_mat.aff12.1D and BBB_mat_INV.aff12.1D. The latter is just the inverse of the former.

To see where the data in AAA would actually sit in scanner space when obliquity is applied, you could create the dataset CCC.nii.gz (which will be regridded during the application of the obliquity, and therefore blurred a bit):

3dWarp -deoblique -prefix CCC.nii.gz AAA.nii.gz 

Applying matrices from obliquity_remover.py:

First, if you want to take the dataset from which you purged obliquity (BBB.nii.gz) and send it to the location where would be if that purged obliquity were applied, you could do this to create a new dset DDD.nii.gz:

3dAllineate                          \
   -1Dmatrix_apply  BBB_mat.aff12.1D \
   -source          BBB.nii.gz       \
   -prefix          DDD.nii.gz

After this, you could overlay DDD.nii.gz on CCC.nii.gz, and they should have effectively the same overlap. Neither CCC.nii.gz nor DDD.nii.gz has any obliquity in their headers, and each will have undergone some blurring during the regridding process by which an oblique (=rotational) matrix was applied. It is possible that DDD.nii.gz will be slightly cut off/truncated, depending on the tightness of the original FOV and obliquity details. Also, CCC.nii.gz and DDD.nii.gz might have slightly different values at each voxel, depending on the interpolation kernels used during the regridding. But there you go.

Second, if you want to create a new dset from BBB.nii.gz that actually has its full obliquity again, you can do the following to create EEE.nii.gz. What we are doing is re-calculating the AFNI header attribute, IJK_TO_DICOM_REAL. So, you can take that attribute from BBB.nii.gz and append to it the BBB_mat_INV.aff12.1D matrix (NB: this is the _INV one) and save it to a text file:

cat_matvec -ONELINE \
    BBB.nii.gz::IJK_TO_DICOM_REAL BBB_mat_INV.aff12.1D \
    > BBB_full_mat.1D

Then, you can use 3drefit to update this attribute in dset. Since 3drefit overwrites header info in place, let's make a copy and edit the attribute there, in case we make a mistake or something:

3dcopy BBB.nii.gz EEE.nii.gz
3drefit \
    -atrfloat IJK_TO_DICOM_REAL BBB_full_mat.1D \
    EEE.nii.gz

After that, AAA.nii.gz and EEE.nii.gz should have the same grid setups and the same obliquity values. In fact, running this should show all 1s:

3dinfo -same_all_grid AAA.nii.gz EEE.nii.gz`

meaning all grid attributes, including obliquity, are the same. Note that AAA.nii.gz and EEE.nii.gz should also have the same voxelwise values, as EEE.nii.gz was never interpolated, it just went through a series of header changes: first purging obliquity to create BBB.nii.gz, and then re-inserting an oblique matrix to create EEE.nii.gz.

I will add this kind of info to the obliquity_remover.py help file, so it becomes long enough that no one will read it.

--pt

1 Like

Hi again, @zhengchencai -

Re. your second question, as to what has obliquity or not after running obliquity_remover.py:

Let's assume that AAA.nii.gz has obliquity (so, 3dinfo -is_oblique AAA.nii.gz produces 1).

If you run the following to remove obliquity from AAA.nii.gz and to append it to child dsets X.nii.gz, Y.nii.gz and Z.nii.gz, like with:

obliquity_remover.py                     \
    -inset         AAA.nii.gz            \
    -prefix        BBB.nii.gz            \
    -child_dsets   X.nii.gz              \
                   Y.nii.gz              \
                   Z.nii.gz              \
    -child_suffix  _out

Then I would expect:

  • 3dinfo -is_oblique BBB.nii.gz produces 0, because we purged obliquity there.
  • 3dinfo -is_oblique X_out.nii.gz produces 1, because we appended obliquity there, regardless of whether X.nii.gz was oblique. The same will hold true for the other *_out.nii.gz dsets.

Importantly, the overlap of AAA.nii.gz and X.nii.gz when all obliquity was applied, should still be preserved between BBB.nii.gz and X_out.nii.gz, even though BBB.nii.gz no longer has obliquity (because that amount of rotation was passed along to X_out.nii.gz). The same holds true for the other *_out.nii.gz dsets. The QC images should show that.

Re. the time it took to run:

  • Is that a very high res dset? It might take a bit of time for that, because of a lot of copying and zipping files. That does seem like a long time, though.
  • Re. images: that is weird. If you run a basic @chauffeur_afni -ulay DSET -prefix image_out command on some DSET, does it create the image successfully?

--pt

ps: I did find one bug in an option checker, where I have referring to an incorrect abbreviation. So, if you get an error like NameError: name 'BASE' is not defined, that will be fixed in the next code update...

1 Like

Hi @ptaylor ,

Thanks for the very detailed explanation, now I understand obliquity_remover.py better.

I also figured out why QC was not working (the example I mentioned was 5mm fMRI). It seem it was because I did not install ffmpeg. My XQuartz was up to date and was working fine for AFNI GUI and apqc, but when I tried @chauffeur_afni, it juse kept trying the following many times until too long, and aborted with errors. I guess the QC got stuck on this and took time.

[1] 2297

_XSERVTransmkdir: ERROR: euid != 0,directory /tmp/.X11-unix will not be created.

_XSERVTransSocketUNIXCreateListener: mkdir(/tmp/.X11-unix) failed, errno = 2

_XSERVTransMakeAllCOTSServerListeners: failed to create listener for local

(EE)

Fatal server error:

(EE) Cannot establish any listening sockets - Make sure an X server isn't already running(EE)

[1]  + Exit 1                        Xvfb :669 -screen 0 1024x768x24

 -- trying to start Xvfb :538

I installed ffmpeg then obliquity_remover.py just took 20s.

Thanks again.

Hi, @zhengchencai -

Great, glad to know that was useful, and that the runtime+QC image issue was resolvable.

That is interesting about ffmpeg resolving that error. That is /tmp/.X11-unix is a known (and annoying) issue on Macs, when running Xvfb (which @chauffeur_afni does). There is this troubleshooting advice: @chauffeur_afni -> Troubleshooting advice:

mkdir /tmp/.X11-unix
sudo chmod 1777 /tmp/.X11-unix
sudo chown root /tmp/.X11-unix/

... or also, I think we have found that just opening any X11-based application (such as the AFNI GUI) once will resolve it for the rest of the time until logging out/in again. I don't know why precisely, but there ya go.

--pt

Hi @ptaylor ,

Indeed you are right. It was not solved by ffmpeg, I rebooted and the error was back. Then I recall that I did open AFNI GUI before running obliquity_remover.py. I did the command you sent it worked but the error was back after rebooting, as tmp was gone. I guess I will just set XQuartz as a startup app and close it after login. Tested, it works.

Hi, @zhengchencai -

Ah, that's a great idea to make Quartz a startup app! Nice.

--pt

A post was split to a new topic: obliquity_remover.py crashes