Batch Processing

[Print-friendly version]

In order to get the best performance for all users of the system, it is necessary to strike a balance between running interactive jobs and scheduling jobs to run in batch mode. Obviously many applications, including editors and compilers, must be used interactively. However, once applications (including Matlab scripts) have been developed, we strongly encourage users to run their jobs through the batch system.

With the new compute resources we have removed the requirement for users to choose queues based on resource requirements, and instead require users to make an estimation of what resources they require.

General Queues

Users are not required to choose a queue, the best queue will be chosen automatically . The following are some of the limits imposed on the queues.

All queues have a default time of 30mins and 512 MB of memory. See below to find out how to increase this.

Batch System Commands

The following commands can be used to submit batch jobs and monitor their progress. For further reading material, download the PBS user guide.

Batch System Script

To submit a job to a PBS queue, first a submit 'script' must be established. Say the user want to run Matlab (and their Matlab code is called myCode.m), they could use a script like to following:

#!/bin/bash -l

#PBS -N myMatlabJob
#PBS -l walltime=3:00:00
#PBS -l mem=400mb

module load matlab/2006a
cd $PBS_O_WORKDIR

matlab -nodisplay -r myCode

Line 1: States this script is written in bash. The -l (that's an L not a 1) is required.
Lines 2-4: These are PBS variables which tell PBS some information about the job. In this case, the name of the job will be myMatlabJob, the job will run for a maximum of 3 hours, and will consume 400mb of memory. You can set these variables to what you think are appropriate.
Line 5: This says that we would like to load the module for the 2006 version of Matlab.
Line 6: This tells bash that we wish to change into the directory we are submitting the script from (ie our Matlab files are in the same directory as our PBS script).
Line 7: This is the command to run Matlab.

To submit this script you would type:   qsub filename   (replace filename with the name of the script).