...
- alias: data.marconi.cineca.it
- hostnames and IPs:
- dmover1.marconi.cineca.it - 130.186.17.140
- dmover2.marconi.cineca.it - 130.186.17.141
Marconi100:
- alias: data.m100.cineca.it
- hostnames and IPs:
- dmover1.m100.cineca.it - 130.186.16.214
- dmover2.m100.cineca.it - 130.186.16.215
Galileo100:
- alias: data.g100.cineca.it
- hostnames and IPs:
- dmover1.g100.cineca.it - 130.186.16.212
- dmover2.g100.cineca.it - 130.186.16.213
...
- alias: data.leonardo.cineca.it
- hostnames and IPs:
- dmover1.leonardo.cineca.it - 131.175.44.50
- dmover2.leonardo.cineca.it - 131.175.44.51
- dmover3.leonardo.cineca.it - 131.175.44.52
- dmover4.leonardo.cineca.it - 131.175.44.53
Main features
This transfer service is containerized, and there are many differences between these nodes and the login nodes.
...
Important: When you are authenticated on a datamover, the environment variables $HOME, $WORK and $CINECA_SCRATCH (as well as ~ or * ) are not defined.
This property has 2 side effects:
...
One entered the sftp session, the familiar pwd, cd /path/to/, ls commads commands are available to explore the remote filesystem, together with the sftp command lpwd, lcd /path/to/, lls. You can also transfer data from the sftp session, see the appropriate section below.
At present M100 login nodes are offline, so you need to resort to the sftp command to explore M100 filesystem. For instance:
$ sftp fcola000@data.m100.cineca.it:/m100_scratch/userinternal/fcola000/
Connected to data.m100.cineca.it.
Changing to: /m100_scratch/userinternal/fcola000/
sftp> cd cuda
sftp> ls -l
drwxr-xr-x 3 fcola000 interactive 4096 Feb 11 2021 targets
sftp> pwd
Remote working directory: /m100_scratch/userinternal/fcola000/cuda
sftp>
Available transfer tools
rsync
...
There are 2 possible ways to use scp sftp via datamovers:
- You need to upload or download data FROM/TO your local machine TO/FROM a CINECA HPC cluster
$ sftp <username>@data.<cluster_name>.cineca.it:/absolute/remote/path/to/
sftp> put relative/local/path/to/file
Uploading /absolute/local/path/to/file to /absolute/remote/path/to/file
file 100% 414 365.7KB/s 00:00
sftp> get relative/remote/path/to/file
Fetching /absolute/remote/path/to/file to file
file 100% 1455KB 19.0MB/s 00:00
sftp> - You need to transfer files between 2 CINECA HPC clusters
$ ssh -xt <username>data<username>@data.<cluster_name_1>.cineca.it sftp <username>@data.<cluster_name_2>.cineca.it:/absolute/path/to/
...
Datamover nodes of each CINECA HPC cluster host the GridFTP servers. On the datamover we have enabled the authentication via ssh as an alternative to the x509 certificate. Of course, you have to keep in mind that the SSH authentication is based on the ssh certificate, obtained via 2-Factor authentication.
Transfer data between 2 CINECA HPC clusters
...
Please refer to GridFTP protocol and globus-url-copy
...
CAVEAT: at present the gsiftp mode is not available on Leonardo.
Transfer data FROM/TO local machine TO/FROM a CINECA HPC cluster
...
Please note that authentication via ssh certificate is not available on Globus onlineclient.
Additional transfer modes will be explored and reported in the next future.