Link to the new User Guide https://docs.hpc.cineca.it/index.html

The Software Catalog

CINECA offers a variety of third-party applications and community codes that are installed on its HPC systems. Most of the third-party software is installed using software modules mechanism (see The module command section). Information on the available packages and their detailed descriptions are organized in a catalog,  divided by discipline (our web site) by selecting "software" and "Application Software for Science". The catalog is also accessible directly on HPC clusters by using the commands: module  or modmap  (see next section).

The module command

All softwares  installed on the CINECA clusters are available as modules . As default, a set of  basic modules are preloaded in the enviroment at login. To manage modules in the production enviroment, the user can execute the command module  with a variety of options. A short description of the most useful module command usage is reported in the following table.

module avail

module load <appl>

module load autoload <appl>

module help <app>

module list

module purge

module unload <app>

Customization of your account by installing fresh softwares

To assist users in customizing their production environment by installing fresh software, we offer a powerful tool named Spack. Spack is a multi-platform package manager that facilitates the easy installation of multiple versions and configurations of software. This tool is currently available on MARCONI, GALILEO100, and LEONARDO. Below, you will find a step-by-step guideline to install new software or a new release of existing software using Spack. For a comprehensive and detailed guide, please refer to the official Spack documentation.

Loading the spack module available on the cluster

We provide a module to load a pre-configured Spack instance:

$ modmap -m spack 
$ module load spack/<version>

By loading a spack module, the setup-env.sh  file is sourced. Then $SPACK_ROOT  is initialized to /cineca/prod/opt/tools/spack/<vers>/none , spack command is added to your PATH, and some nice command line integration tools too.
The directory /spack-<version>  is automatically created into a default space, and it contains some sub-directories created and used by Spack during the phase of the package installation. When you load the spack module, the list of the sub-directories is shown, with the full paths: on MARCONI and GALILEO100, the default area is $WORK/$USER , while on LEONARDO this is $PUBLIC . You will find then, for example on LEONARDO:

• sources cache: $PUBLIC/spack-<version>/cache
• software installation root: $PUBLIC/spack-<version>/install
• modulefiles location: $PUBLIC/spack-<version>/modules
• user scope:  $PUBLIC/spack-<version>/user_cache

For MARCONI and GALILEO100 users, notice that $WORK  space will be removed at the end of the corresponding project. If you want to define different paths for cache, installation, modules and user scope directories, please refer to Spack manual.

Listing the software packages available to be installed

You can check if the software package you want install is available via spack with the comand spack list, which prints out a list of all of the packages Spack can install. You can also specified the name of the package:

$ spack list <package_name>

or

$ spack list | grep <package_name>

Providers

In the spack environment, "virtual packages" are defined (e.g. mpi), which are provided by multiple specific packages (e.g. intel-oneapi-mpi, openmpi, ...). The list of the available virtual packages is given by

$ spack providers

List the packages that are able to provide a specific virtual package

$ spack providers <virtual_package_name>

e.g $ spack providers mpi

(e.g. intel-oneapi-mpi and openmpi packages provide the mpi virtual package; netlib-lapack and openblas provide the lapack virtual package).

Variants and dependencies

If the package of your interest is available to be installed via spack, look at its build “variants” that yon can switch in/off or customize, and  the  “dependencies” that will be used for the building, linking and running phases and you are able to personalize:

$ spack info <package_name>

Listing the software packages installed from Cineca staff via spack

Cineca staff has already installed through the spack module that you loaded a suite of compilers, libraries, tools and applications and you can use them to install additional software (it is strongly recommended). List the packages already installed

$ spack find

Check if a specific package is already installed or what packages have been already installed to provide a specific virtual package (e.g mpi)

$ spack find <package_name>  
$ spack find <virtual_package_name>

List the packages already installed  and see e.g. the used variants (-v), dependencies (-d), the installation path (-p) and the hash (-l). The meaning of the hash is discussed in the next paragraph.

$ spack find -ldvp <package>

You can also list the packages already installed with a specific variant

$ spack find -l +<variant>
e.g. $ spack find -l +cuda

or which depends on a specific package (e.g openmpi) or a generic virtual package (e.g. mpi)

$ spack find -l ^<package_name>
e.g. $ spack find -l ^openmpi
e.g. $ spack find -l ^mpi

or installed with a specific compiler

$ spack find %<compiler>

The list of all the compilers already installed and ready to be used can be seen with

$ spack compilers

Installing a new package

Spec command

In order to install a package with the spack module you have to select for it a version, a compiler, the dependencies and the building variants (we will show how to do it in the following). The combination of all these parameters is the "spec" with which the package will be installed.

If you don’t select any combination during the installation, a default spec is selected from spack. Before installing a package, it is strongly recommended to check the default spec with which the package would be installed:

$ spack spec <package_name>

Each spec is identified by a unique number (e.g. aouyzha), named “hash” and displayed by “-l” (long) option, that distinguishes the several installations of a same package.

See the hash for all the installations of a specific package

$ spack spec -l <package_name>

e.g. see the different hash for openmpi%intel and openmpi%gcc

$ spack spec -l openmpi %intel
aouyzha  openmpi@4.1.1%intel@2021.4.0~atomics+cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~java+legacylaunchers~lustre~memchecker~pmi+pmix~singularity~sqlite3+static~thread_multiple+vt+wrapper-rpath fabrics=ucx schedulers=slurm arch=linux-centos8-cascadelake

$ spack spec -l openmpi %gcc
2ovg6ub  openmpi@4.1.1%gcc@10.2.0~atomics+cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~java+legacylaunchers~lustre~memchecker~pmi+pmix~singularity~sqlite3+static~thread_multiple+vt+wrapper-rpath fabrics=ucx schedulers=slurm arch=linux-centos8-cascadelake

Another suggested option of the "spec" command is " -I" (install), which shows the installation status of the requested package and its dependencies.
See the hash and install status of the selected packages and its dependencies 

$ spack spec -Il <package_name>

e.g. see hash and installing status of openmpi and its cuda dependency

$ spack spec -Il openmpi
-   aouyzha openmpi@4.1.1%intel@2021.4.0~atomics+cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~java+legacylaunchers~lustre~memchecker~pmi+pmix~singularity~sqlite3+static~thread_multiple+vt+wrapper-rpath fabrics=ucx schedulers=slurm arch=linux-centos8-cascadelake
  [-]  rdd7huh      ^cuda@11.5.0%intel@2021.4.0~dev arch=linux-centos8-cascadelake

The installation status is described by the symbol “-” (not installed) or “+/^” (installed/installed from an other user) that precedes the hash of the spec. If the dependency spec (e.g. cuda) of the selected package (e.g openmpi) is not installed (-), it will be by default from spack during the installation of the selected package. The dependency spec installed by default from spack to meet  the building requirement of the selected package is called “implicit” installation, the selected package “explicit” installation.

If a dependency is not fullfilled by the Cineca installation even if it is available, it shows “-” symbol and it will be installed with a different hash, because it's a different spec.

In the following example, openmpi dependency is not fullfilled by the cineca installation (-) even if it is already available with a specific hash as the "spack find -l openmpi"  command  shows. This means that openmpi will be installed again from spack with a different combination of building values and consequently with a different hash (aouyzha). In order to use the Cineca openmpi installation you would have to provide its spec to spack, for example through the hash shown by the following command

$ spack find -l openmpi
==> 1 installed package
-- linux-centos8-cascadelake / gcc@10.2.0 -----------------------
ov3ei7j openmpi@4.1.1# Example: parmetis will be installed with a spec of openmpi that has not been installed from Cineca staff (-)$ spack spec -Il parmetis %gcc@10.2.0 ^openmpi
-   oytpej7  parmetis@4.0.3%gcc@10.2.0~gdb~int64~ipo+shared build_type=RelWithDebInfo patches=4f892531eb0a807eb1b82e683a416d3e35154a455274cf9b162fb02054d11a5b,50ed2081bc939269689789942067c58b3e522c269269a430d5d34c00edbc5870,704b84f7c7444d4372cb59cca6e1209df4ef3b033bc4ee3cf50f369bce972a9d arch=linux-centos8-cascadelake
[-]      aouyzha   ^openmpi@4.1.1%gcc@10.2.0~atomics+cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~java+legacylaunchers~lustre~memchecker~pmi+pmix~singularity~sqlite3+static~thread_multiple+vt+wrapper-rpath fabrics=ucx schedulers=slurm arch=linux-centos8-cascadelake# Example: parmetis will be installed with the spec installed from Cineca staff defined by the hash "ov3ei7j" as the command "spack find -l openmpi" shows$ spack spec -Il parmetis %gcc@10.2.0 ^/ov3ei7j
-   oefbg6x  parmetis@4.0.3%gcc@10.2.0~gdb~int64~ipo+shared build_type=RelWithDebInfo patches=4f892531eb0a807eb1b82e683a416d3e35154a455274cf9b162fb02054d11a5b,50ed2081bc939269689789942067c58b3e522c269269a430d5d34c00edbc5870,704b84f7c7444d4372cb59cca6e1209df4ef3b033bc4ee3cf50f369bce972a9d arch=linux-centos8-cascadelake
 [^]      ov3ei7j      ^openmpi@4.1.1%gcc@10.2.0~atomics+cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~java+legacylaunchers~lustre~memchecker~pmi+pmix~singularity~sqlite3+static~thread_multiple+vt+wrapper-rpath fabrics=ucx schedulers=slurm arch=linux-centos8-cascadelake

Install command

For a specific package you can install the default spec  or a custom spec by selecting a specific version (through @ symbol), and/or a specific compiler (through % symbol), building variants (through +,- or ~,=  symbols) and specific dependencies (through ^ symbol).

Default package installation:

$ spack install <package_name>

or

$ spack install /<package_hash>

You can use <package_name> or /<package_hash> without distinction, in the following for simplicity all the examples are reported with <package_name>.

Overall view of a custom package installation:

$ spack install <package_name>@<version> +/~/<variant> <variant>=<value> %<compiler>@<version> ^<dependency_name>

The available versions of the package, the building variants and the dependencies that you can customise can be shown by executing the command "spack info <package_name>", as described above.

The command "spack compilers" shows all the compilers you can use for your installation and "spack find -l" all the dependencies installed from Cineca staff with their hash.

Remember to install the new package after executing “spack spec” command as described previously and use the Cineca installations for the dependencies if they are available.

If you installed a new compiler, in order to use it remember to add it to your “compilers.yaml” file in the following way:

$ spack install <compiler_package>
$ spack load <compiler_package>
$ spack compiler add
$ spack compilers

Module command and Spack managing

After installing a package, you can load it with the spack command

$ spack load <package_name>

Otherwise you can create a modulefile and load the software package as a module. To create the modulefile of your installed software

$ spack module tcl refresh --upstream-modules <package_name>

Then you can find and load the new modulefile as

$ module load spack
$ module av <package_module>
$ module load <package_module>