Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Login nodes: 4 nodes, icelake no-gpu




Booster

DCGPData Centric

Model

Atos BullSequana X2135 "Da Vinci" blade

Atos BullSequana X2140 three-node blade

Racks

150

Nodes

3456

1536

Processors

32 cores Intel Ice Lake 

Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz

56 cores sockets Intel Sapphire Rapids

2 x Intel CPU, TDB 350W

Accelerators

4 x NVIDIA Ampere GPUs/node, 64GB HBM2

-

Cores

32 cores/node

112 cores/node

RAM

512 (8x64) GB DDR4 3200 MHz

512 (16 x 32) GB DDR5 4800 MHz

Peak Performance

about 309 Pflop/s

9 Pflops/s

Internal Network

NVIDIA Mellanox HDR DragonFly++ 200Gb/s
2 x NVIDIA HDR 2 x 100 Gb/s cards (Booster nodes)
3 x Nvidia HDR100 1 x 100 Gb/s cards (Data Centric DCGP nodes)

Disk Space

106PB Large capacity storage, 620 GB/s (Capacity Tier)
5.4 PB of High performance storage, 1.4 TB/s (Fast Tier)






...

which establishes a connection to one of the available login nodes.  To connect to Leonardo LEONARDO you can also indicate explicitly  the login nodes:

$ login01-ext.leonardo.cineca.it
$ login02-ext.leonardo.cineca.it
$ login05-ext.leonardo.cineca.it
$ login07-ext.leonardo.cineca.it

The mandatory access to Leonardo LEONARDO is the two-factor authentication (2FA). Please refer to this link of the User Guide to activate and connect via 2FA. For information about data transfer from other computers please follow the instructions and caveats on the dedicated section section Data storage or the document  Data Management.

Accounting

The accounting (consumed budget) is active from the start of the production phase. For accounting information please consult our dedicated section.

The account_name (or project) is important for batch executions. You need to indicate an account_no to be accounted for in the scheduler, using the flag "-A"

...

On LEONARDO, as on the other HPC clusters in CinecaCINECA,  a a linearization policy for the usage of project budgets has been defined and implemented.  The The goal is to improve the response time, giving users the opportunity of using the cpu hours assigned to their project in relation to their actual size (total amount of core-hours).

...

In addition to the home directory $HOME, for each user is defined a scratch area $SCRATCH (or $CINECA_SCRATCH), a large disk for the storage of run time data and files.
An new user specific area $PUBLIC is defined on LEONARDO, useful for example to share installations with other users (it is indeed the default directory for SPACK sub-directories, see more details in the dedicated page).
A $WORK  area is defined for each active project on the system, reserved for to all the collaborators of the project.


Total Dimension (TB)

Quota (GB)

Notes

$HOME0.46 PiB50GB per user
  • permanent
  • backed up
  • user specific
$CINECA_SCRATCH41.4 PiBno quota
  • HDD storage
  • temporary
  • , user specific
  • no backup
  • automatic cleaning procedure of data older than 40 days (time interval can be reduced in case of critical usage ratio of the area. In this case, users will be notified via HPC-News).
$PUBLIC0.46 PiB50GB per user
  • permanent
  • user specific
  • no backup
$WORK10 PB
  • permanent
  • , project specific
  • no backup
  • extensions can be considered if needed (mailto: superc@cineca.it)

...

The software modules are collected in different profiles and organized by functional categories (compilers, libraries, tools, applications, ...). The profiles are of two types: “programming” type (base and advanced) for compilation, debugging and profiling activities, and  “domain” type (chem-phys, lifesc, ...) for the production activity. They can be loaded together.

"Base" profile is the default. It is automatically loaded after login and it contains basic modules for the programming activities (ibm, gnu, pgi, cuda compilers, math libraries, profiling and debugging tools, ...).

If you want to use a module placed under other profiles, for example an application module, you will have to previously load the corresponding profile:

...

Almost all the softwares on Leonardo LEONARDO were installed with Spack manager, which loads automatically the possible dependencies, so "autoload" command is unnecessary.

For listing all profiles you have loaded you can use the following command:

...

$ modmap -m <module_name> 

Note: on LEONARDO you can find modules compiled to support GPUs and modules suitable only for CPUs. You can check the compiler in the full name of the module, where the version is specified (e.g. gromacs/2022.3--intel-oneapi-mpi--2021.10.0--oneapi–2023.2.0). Remind that modules compiled with gcc, nvhpc, cuda should be used only on the Booster partition, while modules compiled with intel oneapi are suitable for running on the DGCP partition. Please refer to the specific sections of the two partitons for more details on the available compilers: Booster Programming environment and DCGP Programming environment.

Note also that since the DCGP partition is currently in the pre-production phase, changes in the Module environment may occur. Please read the UserGuide for updates.

 Spack environment

In case you don't find a software you are interested in, you can install it by yourself. 
In this case, on Leonardo  we LEONARDO  we offer the possibility to use the “spack” environment by loading the corresponding module. Please refer to the dedicated section in UG2.6: Production Environment

Please note that we are still optimizing Leonardo LEONARDO software stack, and more installations may be added/replaced. Always check with "module av" (the hash in the module name can change).

...