blob: 5c9a93651ff6b3b1193f7f1798be140a6aa3b73f [file] [log] [blame]
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<title>How to Run PBS Jobs in PTP</title>
<link rel="stylesheet" type="text/css" href="help.css">
</head>
<body>
<h1>How to Run PBS Jobs in PTP</h1>
<h2>
<a name="Introduction"></a>Introduction
</h2>
<p>
The PBS Eclipse plug-in, part of the <a
href="PLUGINS_ROOT/org.eclipse.ptp.doc.user/html/toc.html">
Parallel Tools Platform</a> (PTP), allows you to launch and monitor
applications on local or remote PBS (Portable Batch System)
Distributed Resource Managers.<br>
</p>
<p>
The PBS Resource Managers use the new configurable (JAXB) framework.
(An in-detail explanation of the <a
href="PLUGINS_ROOT/org.eclipse.ptp.rm.jaxb.help/html/toc.html">JAXB
Model</a> can be found in the PTP Developer Guide.) What follows here is
a guide to the PBS-specific launch tabs. There are no special
requirements to run the PBS Resource Managers; consult <a
href="PLUGINS_ROOT/org.eclipse.ptp.doc.user/html/02resMgrSetup.html">Configuring
Resource Managers</a> for general information on resource-manager
requirements and set-up.
</p>
<h2 style="text-align: left;">
<a name="Preferences"></a>Preferences
</h2>
<p>There is currently only one preference option. Since the PBS
Resource Manager is configured from an XML definition file, the file
can be cached the first time you add the resource manager (this is the
default behavior), or you can set it to reread the file every time you
start the Resource Manager. This is useful if you are doing
customization (see below) and are configuring from a special XML file
you have imported into your workspace. This way, if you make changes,
they will automatically be picked up without having to reboot the
application or recreate the Resource Manager.
<p>
<br> <br>
<div style="margin-left: 40px;">
<img alt="Preferences" src="images/00Preferences.png"><br>
</div>
<br>
<br>
<ol>
<li>From the "Eclipse" menu, select "Preferences"</li>
<li>Open Parallel Tools / Resource Managers / Configurable
Resource Manager (JAXB)</li>
<li>Select "Always reload XML from URL" and click "Apply", then
"OK".</li>
</ol>
<h2 style="text-align: left;">
<a name="ProvidedTypes"></a>Preconfigured PBS Resource Manager Types
</h2>
<p>
PTP provides two generic PBS Resource Managers, one which is
configured for batch submissions to the scheduler (<i>PBS-Generic-Batch</i>),
and one for interactive PBS submissions using a pseudo-terminal (i.e.,
"qsub -I"; <i>PBS-Generic-Interactive</i>). The settings in these
configurations should work on most Torque systems and many PBS Pro
setups, though there may be differences in some of the attributes for
job submission which may have to be adjusted for depending on your
local system.
</p>
<br>
<br>
<div style="margin-left: 40px;">
<img alt="PrefiguredTypes" src="images/01PrefiguredTypes.png"><br>
</div>
<br>
<br>
<p>
<b><i>For how to add, remove, start, stop and edit resource
managers, please refer to <a
href="PLUGINS_ROOT/org.eclipse.ptp.doc.user/html/02resMgrSetup.html">Configuring
Resource Managers</a> </i> </b>
</p>
<h2 style="text-align: left;">
<a name="BatchSubmission"></a>Submission Using the PBS Batch Resource
Manager
</h2>
<p>
When you select the <i>PBS-Generic-Batch</i> manager, the Run
Configuration will have a Resources Tab with three sub-tabs available
for use.
<p>
<ul>
<li><b>Basic PBS Settings</b>
</li>
<li><b>Advanced PBS Settings</b>
</li>
<li><b>Import PBS Script</b>
</li>
</ul>
<p>
Each of these is an "independent" environment in the sense that the
specific variables set on one are not necessarily included in the
others, even though their underlying values are shared across all
three tabs; <b><i>the tab from which you select "Run" is the
environment that applies to the job</i> </b>.
</p>
<p>Thus if you use the Advanced Tab and set Job_Name to "pbs1",
then switch to the Basic tab, you will see Job_Name set to "pbs1"
there as well; however, if you launch from Basic, only the variables
seen on the Basic tab will be included in the configuration and script
for the job; any of the variables on the Advanced Tab not on the Basic
Tab will revert to an undefined or default value.</p>
<h3 style="text-align: left;">
<a name="BasicTab"></a>Basic Settings
</h3>
<p>Below is a view of the Basic tab. Job_Name and Wallclock Time
are validated fields, meaning that values with a specific syntax must
be provided (there are defaults); an error message on the top left of
the screen will appear in red in case these are incorrect. Hovering
over the Name label will display a tooltip, if there is one for that
attribute.</p>
<br>
<br>
<div style="margin-left: 40px;">
<img alt="BasicTab" src="images/02Basic.png"><br>
</div>
<br>
<br>
<p>At the left bottom of the tab there are three buttons, "View
Script", "View Configuration" and "Restore Defaults". The first allows
you to inspect what the batch script that is generated from the
filled-in values will look like:</p>
<br>
<div style="margin-left: 40px;">
<img alt="Configuration" src="images/04Script.png"><br>
</div>
<br>
<br>
<p>Similarly, the second button can be used to see the full
configuration underlying this particular tab setting (this is the
"environment" for the resource manager, and is useful to check to make
sure the values you have set on the tab will be correctly reported to
the resource manager):</p>
<br>
<br>
<div style="margin-left: 40px;">
<img alt="Configuration" src="images/03Configuration.png"><br>
</div>
<br>
<br>
<p>"Restore Defaults" sets the values of the variables visible in
this tab back to their defaults, if they have any. This is different
from the "Revert" button on the lower right, which reverts to the
previously saved changes. All changes should be saved using the
"Apply" button next to it.</p>
<h3 style="text-align: left;">
<a name="AdvancedTab"></a>Advanced Settings
</h3>
<p>The Advanced Tab exposes for configuration all the attributes
for job submission as defined in the XML. The viewer allows you to
select the ones that are applicable (only these will appear in the
configuration). The text box above the table/tree containing the
attribute list also displays a string of name=value defintions from
the table (here for viewing convenience only). The table has typed
values; it provides an appropriate editor (drop-down list,
numerical spinner, checkbox, etc.) to fill in the value according to
the value's type; doing "tab" after filling in the value effects the change.
The check box at the top left "Show Only Checked Items" allows you to
filter out unchecked items from the table list.</p>
<br>
<br>
<div style="margin-left: 40px;">
<img alt="AdvancedTab" src="images/05AdvancedSettings.png"><br>
</div>
<br>
<br>
<h3 style="text-align: left;">
<a name="ImportTab"></a>Importing a Predefined Batch Script
</h3>
<p>If you wish to use a custom script that does more than just
launch the application selected from you local or remote workspace,
this tab is where you do it. The "Browse Workspace" button at the top
right takes you first to your current workspace, in case you have
imported a batch script there from either your local machine or a
remote host; however, you can usually navigate from there to other
directories in your local file system to search for a script.</p>
<p>The path of the selected script is displayed in the textbox to
the left of the browse button, and the contents of the script in the
large (scrolling) text area below. The script is not editable there;
to edit, you will need to import it like any other file into your
workspace. The selection can be cleared or cancelled using the "Clear
Script" button.</p>
<br>
<br>
<div style="margin-left: 40px;">
<img alt="Imported" src="images/06Imported.png"><br>
</div>
<br>
<br>
<h4 style="text-align: left;">
<a name="OutputFiles"></a>Reading remote output and error files
</h4>
<p>The JAXB/PBS manager has the capability of allowing you to
stream in to the console the output and error files associated with a
batch job when the job has completed. This is done in XML definition
file for the Basic and Advanced tabs by creating a link from the PBS
batch attributes defining Output Path and Error Path to special
variables understood by the resource manager. If you set those
variables, the change will be conveyed through this linked variable.
If they are not set, the configuration provides a value corresponding
to the default location where PBS would put them.</p>
<p>
<b>NOTE: </b>If you are using the Import Script tab, the two paths
must be manually adjusted to correspond to whatever you have done
inside the script; the default definitions are provided, but these can
be adjusted as necessary. You can also turn off this feature for this
tab by unchecking the "Enable remote output file read" to the left of
the text box.
</p>
<p>If there are files associated with this job, and the job
completes, the "Get Job Error" and "Get Job Output" actions, activated
by clicking row for the job in the monitor job table, can be used to
issue a streaming command which will write the file contents to the
console. If the files do no exist, these actions will be disabled.</p>
<br>
<br>
<div style="margin-left: 40px;">
<img alt="Actions" src="images/08Actions.png"><br>
</div>
<br>
<br>
<h2 style="text-align: left;">
<a name="InteractiveSubmission"></a>Submission Using the PBS
Interactive Resource Manager
</h2>
<p>The Interactive Resource Manager allows you to submit jobs
through a pseudo-terminal. This is done by issuing "qsub -I" (plus any
resource limits defined in the tab) upon the first launch, and waiting
for the interactive job to start. When the job turns from SUBMITTED to
RUNNING (this can be seen both from the reported state in monitor job
table views, as well as by the disappearance of the progress bars for
the submission), the job will then be streamed to the qsub terminal's
standard input, just as you would submit from a shell command-line.
Subsequent submissions will also be streamed to the same interactive
terminal process, for as long as it is running. If the time limit
expires, the next submission will resubmit the terminal request (qsub
-I).</p>
<br>
<p>The Launch Tab for this kind of job closely resembles the Basic
batch tab.</p>
<br>
<br>
<div style="margin-left: 40px;">
<img alt="Imported" src="images/07Interactive.png"><br>
</div>
<br>
<br>
<p>The job output will go to the console when the job starts to
run, as seen here.</p>
<br>
<br>
<div style="margin-left: 40px;">
<img alt="Output" src="images/09Output.png"><br>
</div>
<br>
<br>
<h2 style="text-align: left;">
<a name="Customizing"></a>Customizing the Configuration
</h2>
<br>
<br>
<p>The Resource Manager configuration for PBS can of course be
customized to suit the needs of the systems you habitually work with.
Of course, an understanding of the basic structure and content of the
XML definition is necessary for this, but often it may simply be a
question of changing a few of the provided attribute defaults or
definitions or a special command definition.</p>
<p>The way to provide a specialized XML configuration is by placing
it in a project named "resourceManagers" in your current workspace.
You can import a XML file into your workspace in three different ways:</p>
<br>
<ul>
<li><b>Using Import :: General :: File System</b> for files on
your local machine;<br>
</li>
<li><b>Using Import :: Remote Systems :: Remote file system</b>
for files on a remote machine;</li>
<li><b>Using Import :: Resource Managers :: Resource Manager
Definition from Plug-in</b> for the predefined definitions provided with
the PTP distribution.<br>
</li>
</ul>
<br>
<p>In the first two cases, you must have created the
"resourceManagers" project first; the third wizard asks you if you
wish to create one if one does not already exist. The following
illustrates the last kind of import.</p>
<ul>
<li>Open up your Remote C/C++ perspective containing your
workspace</li>
<li>Right click in the Project Explorer and then select "Import
:: Import..."</li>
</ul>
<br>
<br>
<div style="margin-left: 40px;">
<img alt="Output1" src="images/10Import.png"><br>
</div>
<br>
<br>
<ul>
<li>Choose Resource Managers :: Resource Manager Definition from
Plug-in</li>
</ul>
<br>
<br>
<div style="margin-left: 40px;">
<img alt="Output2" src="images/11Import.png"><br>
</div>
<br>
<br>
<ul>
<li>Select one of the configurations from the drop-down list, and
click "Finish".</li>
</ul>
<br>
<br>
<div style="margin-left: 40px;">
<img alt="Output3" src="images/12Import.png"><br>
</div>
<br>
<br>
<p>The XML file will now appear in an editor in your workspace. The
XML editor which has been included in the PTP all-in-one download
("Eclipse IDE for Parallel Application Developers") does syntax
highlighting and also provides a design tree-view for examining its
structure.</p>
<br>
<br>
<div style="margin-left: 40px;">
<img alt="Output4" src="images/13Import.png"><br>
</div>
<br>
<br>
<hr>
<a href="toc.html">Back to Table of Contents</a>
</p>
</body>
</html>