AFNI version info (afni -ver): Precompiled binary linux_ubuntu_16_64: Nov 7 2023 (Version AFNI_23.3.07 'Septimius Severus')
Dear AFNI team,
I'm struggling with a high-resolution fMRI dataset (172 time points, about 4.8GB) that has been processed with AFNI functions and cannot be read by Freesurfer. So far I have learnt that:
The processed dataset seems to have something wrong with the header, which doesn't seem to stop AFNI functions from editing it but makes it unreadable by Freesurfer (which usually works). Here the output of the header check:
nifti_tool -check_hdr -infiles AFNIprocessed.nii
>
** NIFTI: bad hdr1 fields: dim0, sizeof_hdr = 123, 540
** NIFTI: bad nhdr field: dim[1] = 0
** NIFTI: bad nhdr field: dim[2] = 0
** NIFTI: bad nhdr field: dim[3] = 0
** NIFTI: bad nhdr field: dim[5] = 0
** NIFTI: bad nhdr field: dim[6] = 0
** NIFTI: bad nhdr field: dim[7] = 0
header FAILURE for file AFNIprocessed.nii
The dataset before running any AFNI function on it was ok.
nifti_tool -check_hdr -infiles original.nii
> header IS GOOD for file original.nii
In particular, I have run 3dDespike followed by many others (but the image seems wrong already after 3dDespike). Other functions like 3dcalc can fix the header if I reduce the number of time points considerably, e.g.:
3dcalc -a AFNIprocessed.nii'[0..85]' -expr 'a' -prefix AFNIprocessed_0to85.nii
nifti_tool -check_hdr -infiles AFNIprocessed_0to85.nii
> header IS GOOD for file despiked_0to85.nii.gz
Using 3dcalc without reducing the number of time points (using [0..171]) does not solve the problem.
I realized that valid images (e.g. the original one or the output of 3dcalc with a reduced number of time points) have nifti-1 format, while the one with the header failure are nifti-2 (I checked this using "nifti_tool -disp_hdr -infiles $myfile"). I have read that nifti-2 was introduced to work with large files, so I guess it makes sense that my big file is given nifti-2 format but not my smaller file reduced with 3dcalc; however the original file right before AFNI has nifti-1 format (and is valid), so I guess it should be possible to keep a big file in nifti-1 format.
At this point, my hypothesis is that, for some reason, the nifti-2 format assigned by AFNI is disrupting important header information to the point that other software packages cannot read the image (the error from Freesurfer is: "error: niiRead(): bad number of dimensions (31488) in AFNIprocessed.nii"). I have used Freesurfer in many images processed by AFNI before and never encountered this error. The only thing I can think of is that I updated AFNI about a month ago (I installed it four years ago and never updated it until now), I'm not sure if the issue could have anything to do with this update. Has AFNI made any change regarding nifti-1/nifti-2 in the past couple of years?
I have tried 3dAFNItoNIFTI to convert my processed file to nifti-1, but the output is still in nifti-2 and still has a wrong header and cannot be read by freesurfer.
Is there any solution for this? either fixing the header -maybe applying the header of the original image?- or converting the nifti-2 to nifti-1, which I think should fix the issue? For now, I'm planning on just splitting the file and merge the results at the end, but I'm very curious to know if there is a real solution for it.
Just to be positive, does that ubuntu AFNI package come from our website, or is it from a Docker container? Or was it compiled locally, e.g. via cmake?
Looking into this a little bit, I see that the -check_hdr option has no NIFTI-2 method, it assumes NIFTI-1 format. So those 'bad nhdr' and 'header FAILURE' errors are actually irrelevant.
If possible, would you send me the NIFTI headers, one of a "good" dataset, the other of the dataset that fails with FS? I will send my email address via a message. But please run something like:
And then please send me the actual header file, e.AFNI.hdr.
And please do that for a before-and-after picture, if possible, one that FS would be happy with, one that it would not be.
A little comment is that the output of 3dDespike will default to float format. So it is possible that the dataset size will double there (from short ints to floats).
AFNI was compiled locally (it was a long time ago but I guess via cmake), thank you for starting the debugging.
This is my current output of "afni_system_check.py -check_all":
-------------------------------- general ---------------------------------
architecture: 64bit ELF
cpu type: x86_64
system: Linux
release: 5.4.0-150-generic
version: #167~18.04.1-Ubuntu SMP Wed May 24 00:51:42 UTC 2023
distribution: debian buster/sid
number of CPUs: 12
apparent login shell: bash
shell RC file: .bashrc (exists)
--------------------- AFNI and related program tests ---------------------
which afni : /home/patricia/abin/afni
afni version : Precompiled binary linux_ubuntu_16_64: Nov 7 2023
: AFNI_23.3.07 'Septimius Severus'
AFNI_version.txt : AFNI_23.3.07, linux_ubuntu_16_64, Nov 07 2023, official
which python : /usr/local/bin/miniconda/envs/cpac/bin/python
python version : 2.7.14
which R : /usr/bin/R
R version : R version 3.6.3 (x86_64-pc-linux-gnu)
thank you very much for your quick feedback, I'm sending you the headers via e-mail. The error also occurs when I simply use 3dcopy to copy the original file, so, for fair comparison I'll send you the headers of the original file and its 3dcopy version.
By the way, indeed running nifti_tool -check_hdr -help_hdr2 -infiles AFNIprocessed.nii does not return an error (I didn't know that the file had nifti-2 format before running the check-hdr and didn't know that the flag mattered, thanks!). Still, based on my tests, it seems that Freesurfer complains with these nifti-2 headers only. Looking forward to knowing the problem a little better :-)
Okay, thanks, and thanks for the comment about -help_hdr2, I will try that with your data later tonight (hopefully). I had forgotten some of these details about nifti_tool, despite having written it. Such is life. :)
Regarding cmake, we have recently come to find that our cmake build links BOTH nifti-1 and nifti-2 libraries to our AFNI programs. Not only should only one be linked, but the nifti-1 version seems to take priority, even though AFNI actually requires the nifti-2 library. That is to say, cmake binaries are not so reliable when dealing with NIFTI right now.
If you are on an Ubuntu system, I would strongly suggest going with our standard ubuntu package or your own make build (not cmake build) of it.
Thank you very much for sharing the comment about the nifti libraries, that's interesting.
I'm seeing "Precompiled binary linux_ubuntu_16_64: Nov 7 2023 (Version AFNI_23.3.07 'Septimius Severus')" in my system, does it mean that I used the standard ubuntu package that you referred to? (is there a way of checking how AFNI was installed?)
However, I'm pretty sure I didn't have this problem several months ago, so I'm still tempted to think that the problem might be linked to AFNI's update rather than AFNI's initial installation. Unfortunately I don't remember my previous AFNI version which seemed to have no issues when combined with Freesurfer (probably the most recent version available in 2020), I updated it a few weeks ago running "@update.afni.binaries -defaults -do_extras" to be able of using your QC function ap_run_simple_rest.tcsh. Not sure if that matters, but I also noticed that I needed to link the libgsl.so.19 library afterwards ("sudo ln -s /usr/lib/x86_64-linux-gnu/libgsl.so.27 /usr/lib/x86_64-linux-gnu/libgsl.so.19").
Backing up here, it sounds like you are not using cmake binaries (@update.afni.binaries would not give them, and the afni -ver output should be different). And thank you for the dataset links. I only downloaded enough to pull out the headers, and nothing stands out from them.
I expect that FreeSurfer reads NIFTI-2, but I am not positive. That seems to be the main difference between the datasets, the new one is in NIFTI-2 format (because it got so big).
I might make a change to let such output stay as NIFTI-1, which should help. But it would be good to verify whether FS can read NIFTI-2 in any case.
Anyway, I will get back to you about those things in a bit, when I am able to get to them.
thank you very much, if everything is alright with the files I will contact the FS team then. I thought it was an issue generated solely by AFNI because I was able of finding a failure in the header, but that was before knowing about the nifti-2 flag. Of course you don't need to test it yourself, but if you have access to FS and are curious, I think any function would return the error for my AFNI-processed file. E.g.: mri_info $file
(mri_info is the equivalent of 3dinfo). The output is: error: No such file or directory error: niiRead(): bad number of dimensions (31488) in $file*
Of course I'll have a test myself if we manage to have a nifti-1 output (thanks in advance!). I'll also contact FS to see what they can say about the nifti-2 cases (if that's actually the reason why FS functions fail).
I am looking more closely at this now. It is not clear whether FreeSurfer can read NIFTI-2. So I may make a change to allow such a dataset to stay as NIFTI-1. Hopefully I will get to that reasonably soon...
Thank you so much, that would be really helpful for me, as I like to do the pre-processing in AFNI (you offer so many functions with so many options and usually really well documented with lots of examples, thanks for this!) but I am more used to Freesurfer to work in surface space / visualize surface data.
I asked about the capabilities of Freesurfer to read nifti-2 in their mailing list but I haven't received any response yet.
For now, I'm using 3dcalc to segment my pre-processed AFNI data so that these smaller 4D volumes are saved as nifti-1. I'm working with this in Freesurfer and will merge the final results at the end, it's a bit longer but it will work.
I made a change for this, and it should be done building (and distributing) within a couple of hours (if you compile with build_afni.py, it is available now).
So I took out an nvox test for going to NIFTI-2, which should result in your output staying as NIFTI-1. Also, I added an environment variable (AFNI_NIFTI_WRITE_TYPE) to control the NIFTI type of the output (set it to 1 or 2, as you see fit). You should not need the variable for the situation you are in, but it is there.
Anyway, please give it a try and let me know how it goes, and certainly if you have any questions about it.
Thank you so much, I just updated AFNI and tried a simple 3dcopy on a nifti-2 file, the new file ("AFNIfile_cp.nii.gz" below) can be read by FS now. It also returns the following:
Just to add to the thread solution since they responded to Patricia's query on the mailing list: Freesurfer reportedly only handles NIFTI-1 at the moment:
Thank you for this update! (for some reason I wasn't able of seeing the response at the FS forum before)
The
National Institute of Mental Health (NIMH) is part of the National Institutes of
Health (NIH), a component of the U.S. Department of Health and Human
Services.