...
Portable Batch System (or simply PBS) is the name of a computer software that performs job scheduling. Its primary task is to allocate computational tasks, i.e., batch jobs, among the available computing resources. At present, PBS is the scheduling system of GALILEO, PICO and MARCONI. The pdf version of the "PBSpro 13 user manual" in on this portal.
...
Running applications using PBS
With PBS you specify the tasks to be executed; the system takes care of running these tasks and returns the results to you. If the available computers are full, then PBS holds your work and runs it when the resources are available.
With PBS you create a batch job which you then submit to PBS. A batch job is a file (a shell script under UNIX) containing the set of commands you want to run. It also contains directives which specify the characteristics (attributes) of the job, and resource requirements (e.g. number of processors and CPU time) that your job needs.
Once you create your PBS job, you can reuse it if you wish. Or, you can modify it for subsequent runs.
For example, here is a simple PBS batch job to run a user's application by setting a limit (one hour) to the maximum wall clock time, requesting 1 node with 1 cpus:
#!/bin/bash #PBS -A <account_no> (only for account based usernames) #PBS -l walltime=1:00:00 #PBS -l select=1:ncpus=1 # ./my_application
PBS provides two user interfaces: a command line interface (CLI) and a graphical user interface (GUI). The CLI lets you type commands at the system prompt. Only the CLI is presented here, if you are interested in the GUI, please refer to the official documents.
PBS has been configured differently on the various systems reflecting the different system features. Please refer to the specific system page for more detailed information.
PBS commands
The main user's commands of PBS are reported in the table below: Please consult the man pages for more information.
| qsub | Submit a job |
| qstat | Status job, queue, Server |
| qdel | Delete job |
Submit a job:
> qsub [opts ] job_script > qsub -I [opts] -- /bin/bash (interactive job)
The second command is related to a so-called "Interactive job": sets job's interactive attribute to TRUE. The job is queued and scheduled as any PBS batch job, but when executed, the standard input, output, and error streams of the job are connected to the terminal session in which qsub is running. When the job begins execution, all input to the job is taken from the terminal session. Use CNTR-D to close the session.
Displaying Job Status:
> qstat (lists all jobs)
> qstat -u $USER (lists only jobs submitted by you)> qstat <job_id> (only the specified job)> qstat -f <job_id> (full display of the specified job)
Displaying Queue Status:
> qstat -Q > qstat -Qf <queuename> (Long format of the specified queue)
Delete a job:
> qdel <jobID>.io01 (Galileo)
> qdel <jobID>.node001 (Pico)
> qdel <jobID>.r000u17l01 (Marconi)
> qdel <jobID> (all clusters)
More information about these commands are available with the man command
The User Environment
There are a number of environment variables provided to the PBS job. Some of them are taken from the user's environment and carried with the job. Others are created by PBS.
All PBS-provided environment variable names start with the characters PBS_. Some are then followed by a capital O (PBS_O_) indicating that the variable is from the job's originating environment (i.e. the user's).
The following short example lists some of the more useful variables, and typical values (for instance on Galileo):
PBS_JOBNAME=jobb PBS_ENVIRONMENT=PBS_BATCH PBS_JOBID=453919.io01 PBS_QUEUE=shared PBS_O_WORKDIR=/gpfs/scratch/usercin/aer0 PBS_O_HOME=/plx/usercin/aer0 PBS_O_QUEUE=route PBS_O_LOGNAME=aer0 PBS_O_SHELL=/bin/bash PBS_O_HOST=node166.galileo.cineca.it PBS_O_MAIL=/var/spool/mail/aer0 PBS_O_PATH=/cineca/bin:/galileo/cineca/sysprod/pbs/default/bin: ...
There are a number of ways that you can use these environment variables to make more efficient use of PBS. For example PBS_ENVIRONMENT can be used to test if we were running under PBS. Another commonly used variable is PBS_O_WORKDIR which contains the name of the directory from which the user submitted the PBS job
(NOTE: PBS executes the job scripts in the $HOME directory by default!)
PBS Resources
A job requests resources through the PBS sintax; PBS matches requested resources with available resources, according to rules defined by the administrator; when resources are allocated to the job, the job can be executed.
There are different types of resources, i.e. server level resources like walltime, and chunked reources like number of cpus or nodes. Other resources may be added to manage access to software resources for example, particularly when resources are limited and lack of their availability leads to aborting the jobs when they are scheduled for execution. Details may be found in documentation (module help) of the interested applications.
The sintax of the request depends on which type is concerned:
#PBS -l <resourse>=<value> (server level resources, e.g. walltime) #PBS -l select=[N:]chunk[+[N:]chunk ...] (chunk resources, e.g. cpus, gpus,mpiprocs)
For example:
#PBS -l walltime=10:00 #PBS -l select=1:ncpus=1
Moreover, resources can be required in one of two possible ways:
1) using PBS directives in the job script
2) using options of the qsub command
...