...
System Architecture
Compute Nodes: There are 516 IBM NX360 M5 16-core compute nodes. Each one contains 2 Intel(R) Xeon(R) Hashwell 8-core E5-2630 v3 processors, with a clock of 2.40GHz. All the compute nodes have 128 GB of memory. 384 compute nodes are equipped with 2 accelerators (Intel Xeon Phi 7120P) for a total of 768 accelerators in the whole system.
Login and Service nodes: 8 Login & Viz node NX360M5 are available, equipped with 2 nVidia K40 GPU each. 8 service nodes NX360M5 for I/O and cluster management.
All the nodes are interconnected through a Infiniband network, with OFED v1.5.3, capable of a maximum bandwidth of 40Gbit/s between each pair of nodes.
Accounting
For accounting information please consult our dedicated section.
Budget Linearization policy
On MARCONI a linearization policy for the usage of project budgets has been defined and implemented. For each account, a monthly quota is defined as:
monthTotal = (total_budget / total_no_of_months)Starting from the first day of each month, the collaborators of any account are allowed to use the quota at full priority. As long as the budget is consumed, the jobs submitted from the account will gradually lose priority, until the monthly budget (monthTotal) is fully consumed. At that moment, their jobs will still be considered for execution, but with a lower priority than the jobs from accounts that still have some monthly quota left.
This policy is similar to those already applied by other important HPC centers in Europe and worldwide. The goal is to improve the response time, giving users the opportunity of using the cpu hours assigned to their project in relation of their actual size (total amount of core-hours).
Disks and Filesystems
The storage organisation conforms to the CINECA infrastructure (see Section "Data storage and Filesystems" ). In addition to the home directory ($HOME), for each user is defined a scratch area $CINECA_SCRATCH, a large disk for storing run time data and files. $WORK is defined for each active project on the system, reserved for all the collaborators of the project. This is a safe storage area to keep run time data for the whole life of the project.
$DRES points to the shared repository where Data RESources are maintained. This is a data archive area availble only on-request, shared with all CINECA HPC systems and among different projects.
$DRES is not mounted on the compute nodes. This means that you can't access it within a batch job: all data needed during the batch execution has to be moved on $WORK or $CINECA_SCRATCH before the run starts.
Since all the filesystems are based on gpfs (General Parallel FIle System), the usual unix command "quota" is not working. Use the local command "cindata" to query for disk usage and quota ("cindata -h" for help):
> cindata
...