Dear AFNI experts,
I am a member of the HPC support team at the University of Innsbruck, Austria, and I would like to install AFNI on a shared multiuser multipurpose compute cluster for a small research group that is currently being established, but may grow over time.
several installation methods (OS-based, Conda, and containerized) are described, but I cannot find an overview describing whether these alternatives are equivalent. My goal is a complete installation suitable for a possibly heterogeneous group of users, whose particular needs are unknown at this point in time.
Since our cluster is used by many disciplines, I'd prefer the conda-based or a containerized installation (using the Singularity container runtime environment), because these will hopefully result in self-contained installations with no side-effects outside the installation target directory on a shared file server.
(which would be my number one preference, because we have a miniconda infrastructure in place) the suggestions what to install are made under the pretext of highly individual needs, which obviously is somewhat contrary to my goal of offering a generic AFNI installation suitable for a reasonably wide range of users.
Is there by any chance a list of suggested Conda packages that will result in a "complete" AFNI installation in the above sense? Or else, is a generic multiuser installation a bad idea and I should encourage my users to take control of their installations?
Many thanks in advance, Michael Fink
P.S.: an interesting alternative would be to build AFNI from source using the Spack software management tool ("spack dot io" - unfortunately, this message board lets me add only two links per message), which has become the preferred deployment method in many large HPC sites in the recent years. So far, no AFNI installation shows up in the latest Spack package list.
Spack would allow a fully automated build, optimized for the desired target microarchitecture, but adding a new package requires some effort preparing a package recipe file describing all software dependencies and deviations from standard installation procedures. Are you aware of any activities towards this end?
To start, what OS is your system? The Biowulf HPC (which has AFNI installed) at NIH just moved to Rocky Linux, for example, after being CentOS.
I think in general we have tried to write the installation instructions for each OS to have as many dependencies come from the OS's mainstream package managers---Aptitude on Ubuntu, homebrew on macOS, yum on Fedora, etc.---and then to use other other frameworks for remaining dependencies---such as conda/miniconda most recently.
Note that every OS presents challenges for keeping uptodate in various ways---macOS changes weird underlying things, Aptitude's default Python-matplotlib version is too old on Ubuntu 20.04, etc. What you see on those instruction pages tries to balance each of installation (really!) with stability and installability.
I am not sure if it is possible to get everything needed on an OS through miniconda---again, "what is needed" depends often on the baseline OS---but it is possible to try. We have the system setup check:
that is good at evaluating is everything necessary appears to be present. All the setups should be "equivalent" in the sense of passing the system check, so that the set of AFNI programs can run. Those different setups won't be bit-flip identical, though, of course. And even the underlying dependencies create differences---my goodness, things change in Python quickly, for one.
Probably the best (in the sense of most complete and explicit) list of dependencies to start with would be a modern Ubuntu one, because the installed OS seems pretty plain, and then the things listed in the miniconda instructions, under #7 here.
But Rick might (likely) have better ideas for approaching this.
Many thanks, in particular for the pointers! This clarifies a lot. We are on Rocky 8, we try to keep OS-installed dependencies minimalistic, and installed software goes to a file server.
Given all that, this appears to be a natural use case for setting up AFNI in an Ubuntu 22.04 Singularity container. I'll be glad to post results of my efforts.
For reference, there is a Rocky 8 distribution package, https://afni.nimh.nih.gov/pub/dist/tgz/linux_rocky_8.tgz, with a list of dnf requirements at https://github.com/afni/afni/blob/master/src/other_builds/OS_notes.linux_rocky_8.txt.
And actually, aside from R-devel, where it is needed, the -devel suffix is not needed on the other packages. This dnf list is for compiling, not just running.
I believe our HPC team still uses a nightly cron job to run something along the lines of:
@update.afni.binaries -package linux_rocky_8 \
Then they have users run
module load afni to put it in the PATH. That might be an option for you. They also snapshot some of the AFNI versions when particular user groups want static versions.