When dealing with data produced or used by HPC applications, you should keep in mind two separate issues: where to put them and how to move them. This means "Data Storage" and "Data transfer".

Where: The management of a large amount of data (BigData) is not a trivial task: you can use a normal FileSystem (FS) based area or you should consider to describe your data with metadata, in order to better organize them and share among your colleagues (REPO).

How: The transfer of big data sets from or to an HPC cluster with classical tools like scp is not straightforward because various constraints could limit bandwidth and execution time.



If your data stored on Cineca resources belong to special personal data categories (as for example human genomic data), please consider implications related to GDPR (General Data Protection Regulation), entered into application as of May 2018. Please read carefully our disclaimer!

  • No labels