...
- temporary (data are cancelled after a given period);
- permanent (data are never cancelled or cancelled only a few months after the "end" of the project);
...
$HOME: permanent/backed up, user specific, local
This $HOME is a local area where you are placed after the login procedure. It is where system, and user applications store their dot-files and dot-directories (.nwchemrc, .ssh, ...) and where users keep initialization files specific for the systems (.cshrc, .profile, ...). There is a $HOME area for each username on the machine.
This area is conceived to store programs and small personal data. It has a quota of 50 GB. Files are never deleted from this area, moreover. Moreover, they are guaranteed by daily backups: if you delete or accidentally overwrite a file, you can ask our Help Desk (superc@cineca.it) to restore it. A maximum of 3 versions of each file is stored as a backup. The last version of the deleted file is kept for two months, then definitely removed from the backup archive. File retention is related to the life of the username,; data are preserved until the username remains active.
$WORK: permanent, project specific, local
This $WORK is a scratch area for collaborative work within a given project. File retention is related to the life of the project. Files in $WORK will be conserved up to 6 months after the project end, and then they will be cancelled. Please note that there is no back-up in this area.
This area is conceived for hosting large working data files , since it is characterized by the high bandwidth of a parallel file system. It behaves very well when I/O is performed accessing large blocks of data, while it is not well suited for frequent and small I/O operations. This is the main area for maintaining scratch files resulting from batch processing.
There is one $WORK area for each active project on the machine. The default quota is 1 TB per project, but extensions can be considered by the Help Desk (mailto: superc@cineca.it) if motivated. The owner of the main directory is the PI (Principal Investigator) of the project, all . All collaborators are allowed to read/write in there. Collaborators are advised to create a personal directory in $WORK for storing their personal files. By default, the personal directory will be protected (only the owner can read/write), but protection can be easily modified, for example by allowing write permission to project collaborators through chmod command. This second approach does not affect global files security.
...
$CINECA_SCRATCH: temporary , user specific, local
This is a is local temporary storage, like $WORK, conceived for temporary files from batch applications. There are important differences with respect to $WORK area. It is user specific (not project specific), and it can be used for sharing data with people outside your project. By default, file access is open to everyone, in case you need more restrictive protections, you can set them with chmod command.
...
$CINECA_SCRATCH does not have any disk quota. Please be aware that on Galileo Galileo100 and Marconi Marconi100 clusters, in order to prevent a very dangerous filling condition, a 20TB disk quota will be temporarily imposed to all users when the global quota area will reach the reaches 88% of occupancy; this disk quota will be removed when the global occupancy lowers back to normal.
...
Each compute node is equipped with a local storage which whose dimension differs depending on the cluster (please look at the specific page of the cluster for more details).
When a job starts, a temporary area is defined on the storage local to each compute node:
...
However, the directory is removed at the end of the job, hence job's end; hence always remember to save the data stored in such area to a permanent directory in your sbatch script at the end of the run. Please note that the area is located on local disks, so it can be accessed only by the processes running on the specific node. For multinode jobs, if you need all the processes to access some data, please use the shared filesystems $HOME, $WORK, $CINECA_SCRATCH.
...
This filesystem is mounted on login nodes of all HPC clusters in Cineca. Files retention is related to the life of the username: data are preserved until the username remains active.
...
Backup policies
The $HOME filesystem is guaranteed by daily backups. Daily backups guarantee the $HOME filesystem. In particular cases, a different agreement is possible: contact the HPC support (superc@cineca.it) for further details.
The backup procedure runs daily, and we preserve a maximum of three different copies of the same file. Older versions are kept for 1 month. The last version of deleted files is kept for 2 months, then definitely removed from the backup archive.
...
You are strongly encouraged to use these environment variables instead of full paths to refer to data in your scripts and codes data.
Summary
| $CINECA_SCRATCH | $WORK | $DRES |
|---|---|---|
Created when username has granted access. | Created when a project is opened. | Created on request. |
A clean-up procedure is active. | Data are preserved up to few months after the end of the project. | Data are preserved up few months after the expiring date No backup. |
No quota. | Default quota of 1 TB. Motivated requests for quota increase will be taken into account. | Quota based on the needs. |
| By default files are public (read only). The user can change the permission (chmod) and make files private. It is not possible to restrict access to the group (all username usernames share the same mail unix group). | By default files are private. The user can change the permission (chmod) and make files visible (R o R/W) to project collaborators. | Same as $WORK area. |
...
The tool is available in the module cintools, which is automatically loaded in your environment. However, the module can be unloaded as all the other modules (Modules).
...
$WORK and $DRES are environmental variables , automatically set in the user environment.
...
own: rwx
group: rwx
other: -
In this way, all project collaborators, sharing the same project group, can read/write into the project/dres fileset, whereas other users can not.
...
Endianness is the attribute of a system that indicates whether integers are represented from left to right or right to left. At present, all CLusters in CIneca Cineca are "little-endian".
Managing your data
...