Introduction

GridFTP is a very efficient protocol for transferring data, it enhances the standard ftp service making it more reliable and faster. 

The command line tool used is named globus-url-copy, with which is possible transfer data in the following cases:

  • User Local PC <==> CINECA HPC Cluster

  • CINECA HPC Cluster A <==> CINECA HPC Cluster B
  • CINECA HPC Cluster <==> Other site HPC Cluster

Install the standard client on your local workstation

The following steps help you to easily transfer your data from/to CINECA cluster using globus-url-copy. 

Since 2018, it is possible to install the client using those provided by the Grid Community Forum (GridCF), a global community that provides support for core grid software (https://gridcf.org/).

The GridCF is attempting to support a software stack christened the Grid Community Toolkit (GCT). The GCT is an open-source fork of the venerable Globus Toolkit created by the Globus Alliance. The GCT is derived from the Globus Toolkit, but is not the Globus Toolkit. Further, the GridCF is not a part of the Globus Alliance.

Please refer to the Grid Community Toolkit official documentation https://gridcf.org/gct-docs/ for installation notes.

The following guide is valid both for Debian/Ubuntu users and Windows users (only WSL1 and WSL2).

  1.  Download the package epel from the website https://docs.fedoraproject.org/en-US/epel/#_quickstart
    The instructions below refers to the version epel-release-9-7.el9.noarch.rpm, as an example, whose direct link is https://dl.fedoraproject.org/pub/epel/9/Everything/x86_64/Packages/e/epel-release-9-7.el9.noarch.rpm.
  2.  Open a terminal and run the following commands
     $ sudo apt update 
     $ sudo apt-get install alien dpkg-dev debhelper build-essential 
     $ sudo alien <path-to>/ 
     $ sudo dpkg -i 
     $ sudo apt update
     $ sudo apt-get install globus-gass-copy-progs  

Note: before launching the "sudo dpkg" command, check the name of the .deb file in the output of the "sudo alien" command, which prints a string like "epel-release_9-8_all.deb generated".

Otherwise, if you are a RedHat/Fedora user:

  1.  execute the following command to install the client
     $ sudo dnf install globus-gass-copy-progs


Authentication to the service

Generate the ssh certificate using the step client by the command  

     $ step ssh login '<EMAIL USER>' --provisioner cineca-hpc     


For more info please refer to the page How to connect via 2FA.

Use the standard client

On the machine with ssh certificate, you can transfer your data by using the standard client globus-url-copy. 

    $ globus-url-copy -vb -cd -r sshftp://<username>@gftp.<cluster_name_1>.cineca.it:22/absolute/path/from/directory/      sshftp://<username>@gftp.<cluster_name_2>.cineca.it:22/absolute/path/to/


Moreover you can list files in a specific cluster by the command 

    $ globus-url-copy -list sshftp://<username>@gftp.<cluster_name>.cineca.it:22/absolute/path/from/directory/

Transfer data FROM/TO local machine TO/FROM a CINECA HPC cluster


    $ globus-url-copy -vb -cd -r /absolute/path/from/directory/    sshftp://<username>@gftp.<cluster_name>.cineca.it:22/absolute/path/to/

    $ globus-url-copy -vb -cd -r sshftp://<username>@gftp.<cluster_name>.cineca.it:22/absolute/path/from/directory/       /absolute/path/to/

where <cluster_name> can be:

  • marconi
  • g100
  • leonardo

GridFTP TCP Port Range configuration

Please note that GridFTP servers on our clusters are configured to use the port range 20000 - 25000 for the incoming and outgoing connections.

  • No labels