...
UNICORE (Uniform Interface to Computing Resources) offers a ready-to-run Job submission system including client and server software. UNICORE makes distributed computing and data resources available in a seamless and secure way in intranets and the internet (https://www.unicore.eu/).
UCC (Unicore Command Line Client)
The UNICORE Commandline Client (UCC) is a full-featured client for accessing UNICORE services. It requires a Java runtime environment, version 8 11 or later. We recommend using the OpenJDK.
UCC offers * :
- Job submission and management
- Data movement (upload, download, server-to-server copy, etc) using the UNICORE storage management functions and the available data transfer protocols
- Storage functions (ls, mkdir, ...) including creation of storage instances via storage factories
- UNICORE Workflow service support and much more
- .
https://unicore-docs.readthedocs.io/en/latest/user-docs/ucc/manual.html#
Installation and Startup
The following instructions were taken from the UNICORE Command Line Client User guide, for more info please refer to the UNICORE official documentation (http://www.unicore.eu/documentation/).
Prerequisites
- Operating Systems: currently Linux and Microsoft Windows are supported;
- To run UCC, you need a Java runtime version 11 or later (OpenJDK preferred).
Installation
- Download the installation archive
- Untar the tar.gz file and copy it in your home directory
- or in the folder you prefer.
- Download the CA certificates by clicking on "Package with individual certs from CAcerts file in gz format" at this page. Copy in the folder ~/.unicore/certificates/ and untar it.
- Download the following certificate file unicore.hpc.cineca.it.CA.pem and add it to the certificate folder (not mandatory, but avoids "Add exception" requests at each ucc command launch).
- Login to UserDB using OpenID credentials and copy your OIDC token bearer visualized at this page.
- Configure your UCC preferences file (~/.ucc/preferences) in which you include the location of the certificate folder and insert your personal OIDC/OAuth2 token bearer.
When finished, the file should appear as follows:
UI Text Box |
---|
- michele@nmcarpene205872:~$ ls -ltrh ~/.ucc/preferences
-rw------- 1 michele michele 2,4K nov 8 14:40 /home/michele/.ucc/preferences
# User preferences for UCC |
e.....xLTQzMGQtOTBjZC05ZDY4OTc4ODY4MTciLCJleHAiOjE2Njc5MzY0MTEsIm5iZiI6MCw.......cLXA |
<user>/.unicore/certificates/*.pem |
#
The address(es) of the registries to contact |
|
#
|
#################################################################
The token bearer can be visualized from UserDB:
https://userdb.hpc.cineca.it/node/32904
Finally copy CA certificates inside the certificate folder (not mandatory to use the service) from:
https://winnetou.surfsara.nl/prace/certs/
from the link: Package with individual certs from CAcerts file in gz format (SHA1) (PGP)
if you do not do it you can use the UCC client anyway but it will be asked you to accept the CA
after these simple steps you can use the UCC
...
After these simple steps you can use the UCC service.
Useful basic commands
- Connect to the UCC servic
UI Text Box |
---|
$ ./bin/ucc connect -v |
...
|
...
- Lists all available clusters
UI Text Box |
---|
$ ./bin/ucc list-sites -v |
...
|
and of course start remote jobs via SluRM on the HPC infrastructure, also specifying the HPC cluster
...
- List the available storage areas and their UNICORE paths
UI Text Box |
---|
$ ./bin/ucc list-storages https://unicore.hpc.cineca.it:9111/CINECA-G100/rest/core/storages/HOME https://unicore.hpc.cineca.it:9111/CINECA-M100/rest/core/storages/ROOT https://unicore.hpc.cineca.it:9111/CINECA-G100/rest/core/storages/ROOT https://unicore.hpc.cineca.it:9111/CINECA-M100/rest/core/storages/426538bf-72c2-4cd3-ac32-3907b80b94d1-uspace https://unicore.hpc.cineca.it:9111/CINECA-G100/rest/core/storages/7befdf20-fba9-45a8-a5fe-5a92a9789ebd-uspace https://unicore.hpc.cineca.it:9111/CINECA-M100/rest/core/storages/8ad3b031-7144-4368-8d7d-41df51fcd59f-uspace https://unicore.hpc.cineca.it:9111/CINECA-M100/rest/core/storages/395b7fc5-ec8f-4884-87d0-75a3c4552885-uspace https://unicore.hpc.cineca.it:9111/CINECA-M100/rest/core/storages/5c643e6d-5d47-4669-98f2-c8cf461f0559-uspace https://unicore.hpc.cineca.it:9111/CINECA-M100/rest/core/storages/bbc372b3-3377-4a9b-995b-02aa81d2ae97-uspace https://unicore.hpc.cineca.it:9111/CINECA-M100/rest/core/storages/41b59bab-f310-4bad-b49b-65a1328dca6e-uspace https://unicore.hpc.cineca.it:9111/CINECA-M100/rest/core/storages/171bb395-7246-44a6-9b07-f7849d191a01-uspace |
- List files in an area
UI Text Box |
---|
$ ./bin/ucc ls -v https://unicore.hpc.cineca.it:9111/CINECA-M100/rest/core/storages/HOME |
- Copy a local file ON the cluster
UI Text Box |
---|
$ ./bin/ucc cp samples/povray.u -v https://unicore.hpc.cineca.it:9111/CINECA-M100/rest/core/storages/HOME |
- copy a file locally FROM the cluster (notice the "files" folder to be added to the path)
UI Text Box |
---|
$ ./bin/ucc cp https://unicore.hpc.cineca.it:9111/CINECA-M100/rest/core/storages/HOME/files/lista_nodi.txt -v . |
- Start remote jobs via SLURM on the HPC infrastructure (need to specify the HPC cluster with -s <cluster>)
UI Text Box |
---|
$ ./bin/ucc run -v samples/date.u -s CINECA-G100 |
...
...
[ |
...
ucc run] Registry connection status: OK |
To request for new features or customization to be added users should contact the service manager (superc@cineca.it).
...
More features will be added soon and fully documented here.
FAQ
The service fails when try to connect
It may happen that when trying to connect the command "ucc connect -v" gives the following Error
UI Text Box |
---|
[ucc connect] Connecting to https://unicore.hpc.cineca.it:9111/CINECA-G100/rest/core/factories/default_target_system_factory |
It is possible that you OIDC token is expired.
Please login via OpenID on UserDB and download the new OIDC token. Replace it in ~/.ucc/preferences and try again.