Copy of -UNICORE: Job Submission

UNICORE (Uniform Interface to Computing Resources) offers a ready-to-run Job submission system including client and server software. UNICORE makes distributed computing and data resources available in a seamless and secure way in intranets and the internet (http://www.unicore.eu).

URC (Unicore Rich Client)

The client software enables users to create descriptions of work to be performed on the HPC facility, so called ‘jobs’. A single job usually corresponds to the execution of a computer program on one of the available clusters. Once a job has been created, the UNICORE Rich Client can submit it to a selected cluster. The remote execution of the job can be monitored and output files of the executed program can be downloaded to the user’s laptop. In order to accomplish more complex tasks jobs can be embedded into workflows. In our terminology, a workflow is a set of activities (the execution of a single job would be considered an activity), interconnected by transitions that define the order in which the activities must be performed. Workflows can be created and edited graphically. Similar to jobs, they can be submitted to a designated service on the HPC facility which executes them (the UNICORE/X Server and the TSI perl daemon). Workflow execution can be monitored in multiple ways and resulting output files can be downloaded to the local harddisk. Apart from these basic features, the UNICORE Rich Client offers a bunch of additional functions like browsing and monitoring services on the cluster, managing user certificates, and transferring files to and from HPC storages.

Installation and Startup

The following instructions were taken from the UNICORE URC User guide, for more info please refer to the UNICORE official documentation (http://www.unicore.eu/documentation/).

Prerequisites

• Operating Systems: currently Linux and Microsoft Windows are supported;
• Java Runtime Environment: Sun Java 5 or higher is required;
• A valid personal certificate x509. Before accessing UNICORE Service, each user needs to obtain a valid X.509 certificate which is issued by one of the certificate authorities (CAs).

Procedure

• Download the installation archive http://www.unicore.eu/download/ (clients --> rich client --> version that matches your operating system;
• Unzip the archive to the desired location;
• Run the executable called ‘UNICORE_Rich_Client.exe’ (or ‘UNICORE_Rich_Client’, on a Unix/Linux machine).A splash screen will indicate the startup of the client;
• Specify location and passphrase of the keystore file that holds your x.509 certificate.

If you don't have a keystore.jks file containing your certificate/keys you can simply leave it empty and UNICORE will create a demo default one. Then you will be able to add later your personal certificate (also in pkcs12 or pem format) directly from the Keystore view of the URC.

• Add the keystore file hat contains a list of trusted CAs’ certificates (truststore file)

The client presents the x.509 certificate to the server whenever he is asked for authentication. The server then checks whether it trusts the CA that issued the certificate. It does so by searching for the CA’s certificate in a so-called ‘truststore’ i.e. a file that contains a list of trusted CAs’ certificates. If the CA’s certificate is found, it knows it can trust the client. Analogously, the client checks whether it trusts the server. If both checks are successful, a communication channel is created. 

In order to get the truststore file go to the sara web site: https://winnetou.surfsara.nl/prace/certs/ and download the Java keystore (password storepass). Then you will be able to add the trustore file directly from the Truststore view of the URC.

For users that don't already have credentials and are not able to add their personal certificate, the MyProxy Certificate Authority (CA) provides a convenient method for obtaining them. The MyProxy CA issues short-lived session credentials to authenticated users. MyProxy combines an online credential repository with an online certificate authority to allow users to securely obtain credentials when and where needed. Users needs to ask a CINECA for x509 certificate. Once it is obtained, users run myproxy-logon on the CINECA HPC  to authenticate and obtain credentials, including trusted CA certificates and Certificate Revocation Lists (CRLs).

> myproxy-logon -s grid.hpc.cineca.it -l username

 Run the executable called ‘UNICORE_Rich_Client.exe’ (or ‘UNICORE_Rich_Client’, on a Unix/Linux machine) and insert credential from myproxy. Add the keystore file hat contains a list of trusted CAs’ certificates (truststore file) previously downloaded from the sara web site: https://winnetou.surfsara.nl/prace/certs/ with it's password (password storepass).

Base usage guide

When the client is started for the first time, it will display a welcome screen that provides valuable information and helps in making the first steps with the UNICORE Rich Client. If you don't want to read and you prefer to go ahead simply close the window and the URC graphical workbench will appear.

URC workbench

The client’s main window is called the workbench. It has different components which can be opened, closed, resized, re-ordered and even detached from the main window.

Menu bar and tool bar

At the top of the workbench, there is a menu bar from which different pull down menus containing ‘global’ actions can be opened. For convenience, some actions are available via shortcuts from the tool bar just below the menu bar. The items in the tool bar can change depending on the selection of objects in the client, mirroring the fact that different actions can be performed on different objects.

Truststore view

Use this view to add certificates of trusted CAs to the truststore. To this end, import a keystore (‘.jks’ or ‘.p12’) or ‘.pem’ file that contains these certificates. Certificates can also be removed from the truststore. Additional actions allow for opening a detailed certificate description, changing a certificate’s alias (used aliases must be unique) exporting public keys to ‘.pem’ files and setting the keystore password.

 Keystore view

This view is used to manage private keys together with the associated X.509 user certificates

Grid Browser view

This view represents the Grid as a tree structure. The items that form the tree are called ‘nodes’ and represent Grid services and files. There are numerous actions that can be performed on this view or its nodes:

•  Adding registries: For getting started, open the context menu (by right-clicking inside the Grid Browser) and select add Registry . In the appearing dialogue, enter the URL of a registry that serves as an entry point to the Grid: A registry is used for looking up all available services. For each added registry, a new node should appear just below the root.

In order to access to CINECA resources and clusters the proper Registry url must be added:

https://grid.hpc.cineca.it:9111/PRACE/services/Registry?res=default_registry

Users with right permissions and certificates can access trough the graphical interface to the submission service on GALILEO, MARCONI and PICO hpc clusters in CINECA.

•  Refreshing nodes: By double-clicking a node, the represented Grid service is contacted and information about its state is gathered. 

•  Opening the context menu on a selected node: By right-clicking a node, a context menu that contains all available actions for the associated service will appear. For instance, users can create job descriptions for job submission to a target system by selecting the 'create job' action from the target system’s context menu.

•  Administrative actions: The Grid Browser offers additional actions like creating and destroying Grid service instances. 

Job submission and visualization of job outcomes

Now we go ahead to see how to submit simple serial and parallel jobs using UNICORE Rich Client.

Job editor

The UNICORE Rich Client offers graphical editors for setting up job descriptions. Instead of having to edit text-based job descriptions, the user is provided with high level interfaces which are taylored to the applications he wants to execute on remote systems. The client is easily extensible with new application specific user interfaces as new applications are introduced to the Grid environment, new java plugins can be added as jar files into the URC client plugin folder (the so-called GridBeans). Setting up a job description only requires a few simple steps and can be performed within a couple of seconds. The first step is the creation of a job project.

Creating a job project

There are different ways to create a new job project, we're going to see now a very base example:

•  Select File -> New -> Job Project from the menu bar;

•  Open the context menu of the Navigator view and select New ->Job Project;

•  Use the create job item from the context menu of a target system node;

•  Choose the restore Job description item from a job’s context menu in the Grid Browser.

The first three of these options will pop up a series of wizard dialogs which will guide the user through the creation of the job project.

The first step of the wizard is used to choose an application to be run on the target system. In our example, we would like to execute a simple shell script. Therefore, we have selected the Script application.

By pressing the Finish button the new job project is created. Click Next which will take you to the next wizard step. Here, a different name for the project and the job file can be set. The third wizard page allows for selecting a different target system for job submission.The selected target system can also be modified after the project has been created.

When the job is created with the last option, both the target system selection and UNICORE Rich Client user manual application to be run are restored from the server. Therefore, the job creation wizard shows the second wizard page, only (where you can set names for the project and job file).

Budget

In order to select the proper budget the user should set the 'account number' parameter from the same TAB (accountNo).

Username

UNICORE maps the user DN with the first username present inside the Xuudb (the UNICORE Database), if more than one username is associated with the DN the user can select the right username specifying it in the remote login field.

Starting from the URC  6.4.4.v2 release the Security Profile panel has been modified in order to allow users selecting the right username also before starting the job.

Window -> Security Configuration

Add as many novel profiles as you want in the 'Available Security Profiles' window and select the proper user ID to be mapped on the machine. The user DN could be the same or different, depending on the selected user ID.

Finally add a new site to map (for example Grid/PRACE-Registry/*) and choose the desired cluster from the Grid Browser. Now you will be able to perform any operation like file browsing/data staging and job submission using the selected user ID.

Example

In the example below a simple job script has been created to submit over an example Target Site.

#!/bin/bash

hostname

from the script's resource TAB the parameters can be selected, as for example the 'number of cores' requested and number of GPUs.

UNICORE will add right parameters to the job description file and also add correct scheduler directives (as for example #PBS -l select=2:ncpus=12:mpiprocs=12), so the user must only specify the command to execute.

After the job has been submitted the 'stdout' file can be retrieved by the user, this file contains the output of the command. This job simply prints the hostname: "node141"

In the example below instead a parallel mpi job is submitted as a bash script:

module load autoload openmpi

mpirun /bin/hostname

  and this is the output  using 4 cores:

 It is also possible to select a specific Execution Environment (for example MPIRUN-Intel-1.4.2), Execution Environments allow configuring executables paths and executing PRE-COMMANDS (as for example module load) and POST-COMMANDS.

To request for new Execution Environments to be added as well as new Applications, users should contact the service manager (superc@cineca.it).