| <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> |
| <html> |
| <head> |
| <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"> |
| <title>The PTP Configurable Resource Manager</title> |
| <link rel="stylesheet" type="text/css" href="help.css"> |
| </head> |
| <body> |
| <br> |
| <h1>The JAXB Configurable Resource Manager</h1> |
| |
| <h3> |
| <a name="PTP501">PTP 5.0.1 Changes</a> |
| </h3> |
| |
| <p>Aside from some significant bug fixes, there are a number of new |
| features with the 5.0.1 release.</p> |
| <ol> |
| <li>The <code>widget</code> component has been split up into |
| three components: <a href="#Widget"><code>widget</code> </a>, <a |
| href="#ButtonGroup"><code>button-group</code> </a>, and <a |
| href="#Browse"><code>browse</code> </a>;</li> |
| <li><a href="#Action"><code>action</code> </a> push-button |
| functionality has been added;</li> |
| <li><a href="#ControlState"><code>control-state</code> </a> |
| elements have been added to all UI control descriptors;</li> |
| <li>The launch tab <a href="#LaunchTab"><code>import</code> </a> |
| controller type has been modified (largely for maintaining unformity |
| of implementation) to subclass the <code>dynamic</code> controller;</li> |
| <li>Ability to inspect invisible discovered (excluded) properties |
| and attributes via a "View" button (see <a href="#TabController">note</a>) |
| has been added;</li> |
| <li><a href="#Preferences">Preferences</a> have been added to |
| help with the debugging of <a href="#StreamParsers">stream |
| tokenizers</a>;</li> |
| <li>Resource Manager "environment" handling has been totally |
| rewritten in order to support the exclusion of properties associated |
| with invisible or disabled widgets, and to allow for the restoration |
| of their values when re-enabled.</li> |
| </ol> |
| |
| <p>In addition, some tweaking of the SWT "knobs" was necessary to |
| get the XML to reflect more closely the behavior of the Java classes |
| (especially in terms of defaults).</p> |
| <hr> |
| |
| <h2> |
| <a name="Introduction">Introduction</a> |
| </h2> |
| |
| <p> |
| The JAXB Resource Manager plug-ins allow you to launch and monitor |
| applications on local or remote resources using resource managers |
| which are configured from an XML file via JAXB ( |
| <code>javax.xml.bind</code> |
| ) technology.<br> |
| </p> |
| |
| <p>There are two main motivations for providing this class of |
| resource managers:</p> |
| <ol> |
| <li>To allow for maximum adaptability. Often job schedulers (PBS, |
| LSF, LoadLeveler, etc.) or interactive runtime systems (OpenMPI, PE, |
| SLURM, etc.) are set up by system administrators in special or |
| non-standard ways which make it difficult to use a generic tool. The |
| configuration file allows a user or community of users to fit the |
| resource manager to a class of systems, to a single host, or even to |
| special application usage.</li> |
| <li>Building the resource manager and its UI presentation from an |
| XML configuration means that in most cases no special Java coding is |
| necessary. Users should be able to accommodate new systems, at least |
| on the client-end, without writing and loading additional Eclipse |
| plugins. (The only qualification here is that the monitoring |
| component also support that type of scheduler or runtime; see the |
| following paragraphs.)</li> |
| </ol> |
| |
| <p>Additional considerations in designing a generically |
| configurable resource manager were to partition the client |
| functionality so as to eliminate the need for special server-side |
| proxies and to scale more successfully in the updating of job and |
| resource information.</p> |
| |
| <p> |
| To this end, JAXB resource managers now consist of two components, a |
| "control", which governs the configuration, launch and cancellation of |
| individual jobs entirely from the client end, and a "monitor", which |
| displays job status as well as global information about the HPC |
| resource. In most cases, the monitor will be a pre-built type provided |
| by the PTP distribution, implemented using <a |
| href="http://www2.fz-juelich.de/jsc/llview">LLview</a>. Since <b>LLview</b> |
| already supports a good number of the standard scheduler types, adding |
| a new resource manager type will normally entail only the specific |
| configuration of its control part. The default mode of usage is that |
| the client stages the necessary LLview components (mostly Perl |
| scripts) automatically, but in a future release, the monitor will also |
| be capable of connecting directly to a system-wide (web-based) |
| deployment. |
| </p> |
| <p> |
| <b>The following is a guide to the resource manager XML |
| definition. Those interested only in using the JAXB resource managers |
| already provided with the PTP distribution should consult the User |
| pages under the relevant scheduler (currently only the PBS resource |
| managers are JAXB-configurable; please see the <a |
| href="PLUGINS_ROOT/org.eclipse.ptp.rm.jaxb.pbs.doc.user/html/toc.html">PBS |
| Res Mgr help</a>).</b> |
| </p> |
| <hr> |
| <h2> |
| <a name="Customizing"></a><a href="http://download.eclipse.org/tools/ptp/docs/JAXBDemo.pdf">Configuring/Customizing |
| the Resource Manager</a> |
| </h2> |
| |
| <p> |
| We provide here an introductory tutorial slide set demonstrating some |
| simple additions to a pre-existing resource manager definition file. |
| For some example XML illustrating various aspects of the schema |
| (particularly UI features), we also provide a <a |
| href="http://download.eclipse.org/tools/ptp/docs/JAXBExamples_5.0.1.1.zip">.zip file (for PTP release 5.0.1)</a>. |
| </p> |
| |
| <h3> |
| <a name="Preferences"></a>Preferences |
| </h3> |
| |
| <p>The preference options are aids for debugging configuration |
| changes. The first has to do with altering the default behavior of the |
| resource manager, which is to cache its XML definition file; this can |
| be overridden to reread the file every time you start the Resource |
| Manager. This way, if you make changes, they will automatically be |
| picked up without having to reboot the application or recreate the |
| Resource Manager.</p> |
| |
| <p>The other options are useful if you are writing a new stream |
| tokenizer for a command; checking all four options usually produces |
| quite a bit of output, so in that case it may be advisable to log the |
| results to a file rather than stdout (at any rate the latter would be |
| available only if PTP were running as an application within the |
| Eclipse IDE platform, which is not the case for those modifying the |
| XML simply from within the PTP package installation workspace).</p> |
| <br> |
| <div style="margin-left: 40px;"> |
| <img alt="Preferences" src="images/00Preferences.png"><br> |
| </div> |
| <hr> |
| |
| <h2> |
| <a name="XSD"></a><a href="resource_manager_type.xsd">The JAXB |
| Configurable Resource Manager XML Schema</a><br> |
| </h2> |
| |
| <p> |
| The JAXB Resource Manager is <b><i>model-driven</i> </b>; this means |
| that its functioning and appearance are determined by a set of |
| definitions provided via an XML file. What follows is a detailed |
| explanation of the schema governing the resource manager XML |
| definition. |
| </p> |
| |
| <h3> |
| <a name="ResourceManagerData">The Resource Manager Data Type</a> |
| </h3> |
| |
| <img alt="ResourceManagerData" src="images/01rm-data.jpeg" /> |
| |
| <p> |
| The top-level of the definition tree consists of three elements: <b>site-</b>, |
| <b>control-</b> and <b>monitor-data</b>. In addition, a resource |
| manager should be given a name which sufficiently distinguishes it |
| from others of a similar type; e.g., <i>pbs-torque-v_2.3.7_abe</i> is |
| specific to an installation on the host abe, <i>ll-v_4.0</i> suits all |
| installations of LoadLeveler version 4, etc. |
| </p> |
| |
| <p> |
| The <b>site-data</b> element provides an optional place to set fixed |
| or default remote site information. The connection strings are URIs |
| which are specific to the PTP RemoteServices definitions. The scheme |
| for these URIs will usually name the specific remote service (e.g, |
| <code>rse:</code> |
| or |
| <code>remotetools:</code> |
| ; local is simply |
| <code>file:</code> |
| ). The host name and port given here will appear as defaults in the |
| resource manager selection wizard when you create a new connection. |
| </p> |
| |
| <p> |
| The principal section of the schema is devoted to defining the |
| resource manager's <b>control</b> part. The top-level control elements |
| include properties and attributes, files to be staged, job script to |
| be generated (if any), commands specific to the resource manager, and |
| the layout of the Launch Tab. |
| </p> |
| |
| <h4> |
| <a name="Environment">Resource Manager Environment</a> |
| </h4> |
| <p> |
| The resource manager implementation constructs a variable map from the |
| defined properties and attributes which serves as the resource manager |
| "environment". Some elements directly reference these by name; a |
| text/string representation of property or attribute fields can be |
| obtained via the Eclipse variable resolver syntax, with the namespace |
| prefix set to <i>ptp_rm</i>: |
| <code> |
| ${ptp_rm:name#<i>fieldName</i>} |
| </code> |
| , e.g., |
| <code>${ptp_rm:queues#value}</code> |
| (see further <a href="#PropertyAttributeType">below</a> on the |
| specific fields for properties and attributes). |
| </p> |
| <p> |
| All properties and attributes defined in the configuration are mapped. |
| The following hard-coded properties are also added at runtime: <br> |
| <br> |
| <code>control.user.name</code> |
| <br> |
| <code>control.address</code> |
| <br> |
| <code>control.working.dir</code> |
| <br> |
| <code>executablePath</code> |
| <br> |
| <code>progArgs</code> |
| <br> |
| <code>directory</code> |
| <br> <br> In addition, these properties may be linked (see <a |
| href="#PropertyAttributeType">below</a>) to other attributes to |
| furnish the resource manager with an implementation-independent |
| variable for the queue and for any remote output files produced in |
| connection with the job submission. <br> <br> |
| <code>control.queue.name</code> |
| <br> |
| <code>stdout_remote_path</code> |
| <br> |
| <code>stderr_remote_path</code> |
| <br> |
| </p> |
| |
| <h4> |
| <a name="CommandTypes">Resource Manager Command Types</a> |
| </h4> |
| <p> |
| <i>Commands</i> are system calls, either to a local or remote OS, |
| depending on the connection defined for the resource manager. The <b>start-up-</b> |
| and <b>shut-down-command</b>s are arbitrary commands to be run |
| (serially in order) when the resource manager is started or stopped. |
| The <b>submit</b> commands are those used to launch jobs. Currently a |
| configuration may have only a batch or an interactive mode. Thus it |
| may have only two submission modes, a run and a debug, for the given |
| type. In the future we may allow all four to coexist in a single |
| configuration. <b>get-job-status</b> is a user-initiated (on-demand) |
| request to refresh the status information for a submission. Normal |
| (polled) updates, on the other hand, are the responsibility of the |
| <code>monitor</code> |
| component. The status command nevertheless needs to be implemented in |
| most cases, as it will be called internally just after submission. The |
| remaining "-job" commands are operations which can be executed on |
| submitted jobs; with the exception of <i>terminate</i>, the rest of |
| these have to do with schedulers (batch-systems) and do not apply to |
| resource managers which connect to interactive runtime-systems such as |
| OpenMPI or PE. <b>Note</b>: if the submission type is interactive, the |
| <b>terminate-job</b> command usually does not need to be implemented, |
| as the process termination will be handled internally. However, in |
| some cases (such as PBS -I) which require the interactive job to run |
| as a pseudo-terminal, one may need this command in order to force its |
| termination externally. Finally, a <b>button-action</b> is an |
| arbitrary command associated with a button exposed through the |
| <code>Launch Tab</code> |
| (see further <a href="#Action">below</a>). |
| </p> |
| |
| <h3> |
| <a name="ControlType">The Control Data Type</a> |
| </h3> |
| |
| <img alt="ControlData" src="images/02control.jpeg" /> |
| |
| <p>The majority of the XML definition is given over to the set-up |
| of the resource manager control. One can think of this section as |
| having four subdivisions:</p> |
| |
| <ol> |
| <li>Configuration Variable Definitions (the <i>Environment</i>i)</li> |
| <li>Files and Scripts</li> |
| <li>External Commands and their Stream Parsers</li> |
| <li>UI Configuration (Launch Tab)</li> |
| </ol> |
| |
| <p>We will look at these each in turn.</p> |
| |
| <h4> |
| <a name="PropertyAttributeType">1. Property and Attribute Types</a> |
| </h4> |
| <p> |
| A property is any variable necessary for the functioning of the |
| resource manager. Properties often (but not necessarily) are not |
| visible. The value for properties can be any primitive type, or lists |
| or maps of strings. If |
| <code>stdout</code> |
| and |
| <code>stderr</code> |
| from a scheduled job is to be delivered to the client, the properties |
| <i>stdout_remote_path</i> and <i>stderr_remote_path</i> should be |
| included in the resource manager property set. Simarly, the native |
| designation for the selected queue, if used, should be linked to <i>control.queue.name</i>. |
| </p> |
| |
| <p>The distinction between attribute and property is not hard and |
| fast, but the intention is to provide a way of differentiating between |
| external, system-determined variables (attributes) and variables only |
| required by the PTP implementation itself. There are several other |
| elements and fields on the attribute, e.g., for type-specific defaults |
| (min/max, choice). Generally, if the variable is exposed to the user, |
| who should also be provided with a description or instructions for |
| use, then it should be defined as an attribute.</p> |
| |
| <img alt="Attribute" src="images/040attribute.jpeg" /> |
| |
| <p> |
| The <i>visible</i> attribute is a way of indicating to the resource |
| manager that the user will not be directly changing the value of the |
| property or attribute via the user interface. Certain widgets (such as |
| the |
| <code>attribute viewer</code> |
| ) check this to see if the property or attribute should be included |
| automatically in its list. |
| </p> |
| |
| <hr> |
| <h5> |
| <a name="Note1">BEST PRACTICE: VISIBLE vs INVISIBLE</a> |
| </h5> |
| <p> |
| Always mark properties or attributes which are the targets of <i>saveValueTo</i> |
| (see <a href="#Widget">below</a>) as <i>visible</i>="true"; if this |
| value is false, this variable will <i>always</i> be passed to the |
| configuration/environment of the resource manager; selection/exclusion |
| of sets of properties or attributes pertains only to those marked |
| visible. |
| </p> |
| <hr> |
| <p> |
| <i>status</i> is an optional field for capturing the validity of the |
| attribute at runtime or for a particular installation, if such |
| information is made available by the system. The <i>readOnly</i> |
| property can be defined here, but more often will be enforced directly |
| on the widget which is connected to the property or attribute. <i>min</i> |
| and <i>max</i> give the boundaries for integer values, usually |
| represented by spinner widgets. <i>translateBooleanAs</i> applies to |
| boolean values which should be represented as strings other than |
| "true" and "false"; use a comma-delimited pair to indicate the string |
| equivalent for the boolean; e.g., "YES,NO" would mean true yields the |
| first string and false the second. |
| </p> |
| |
| <p> |
| <code>description</code> |
| is meant to be a brief one-sentence definition; most of the time |
| detailed information will go into the |
| <code>tooltip</code> |
| element. |
| <code>choice</code> |
| is a predetermined, fixed set of values to choose from; these are |
| comma-delimited and will appear as the items of a combo-box. |
| <code>items-from</code> |
| instead links the combo items (choice) to another property or |
| attribute value (whose type must be |
| <code>java.util.Collection</code> |
| ). |
| </p> |
| |
| <h4 align="center"> |
| <a name="PAValueLink"> Property and Attribute Values </a> |
| </h4> |
| |
| <p> |
| The untyped |
| <code>value</code> |
| element on properties and attributes is for internal use only; to give |
| a predefined (primitive) value, use the |
| <code>default</code> |
| element along with the <i>type</i> attribute. |
| </p> |
| |
| <p> |
| <code>link-value-to</code> |
| works as follows: the property or attribute gets its value from the |
| property or attribute linked to, unless that property or attribute has |
| an undefined (empty) value; in the latter case, any default defined on |
| the present property or attribute will be used. This feature is |
| primarily intended for defining dependent properties or attributes |
| whose values are not directly modified by the user. |
| </p> |
| |
| <p>Attributes also carry optional validators which allow you to |
| enforce syntactic rules, numerical ranges (each with one or both |
| boundaries; these are OR'd) or file-system semantics on the value when |
| the user enters text.</p> |
| |
| <img alt="Validator" src="images/041validator.jpeg" /> |
| |
| <h4 align="center"> |
| <a name="Regex">Regex and File Match Types</a> |
| </h4> |
| |
| <p> |
| The validator makes use of the |
| <code>regex</code> |
| type (also used in the parser or tokenizer; see <a |
| href="#StreamParsers">below</a>), along with a special |
| <code>file-match</code> |
| type. A regex can be used to match by setting the <i>expression</i> |
| field; to split the segment into tokens on the basis of the |
| expression, set the <i>split</i> field to true (this applies only to |
| the |
| <code>match</code> |
| type discussed <a href="#MatchType">below</a>). The expression should |
| follow the grammar in |
| <code>java.util.regex</code> |
| ; the <i>flags</i> are those made available in the |
| <code>java.util.regex.Pattern</code> |
| class (consult the Javadoc for explanations): |
| </p> |
| <ul> |
| <li>CASE_INSENSITIVE</li> |
| <li>MULTILINE</li> |
| <li>DOTALL</li> |
| <li>UNICODE_CASE</li> |
| <li>CANON_EQ</li> |
| <li>LITERAL</li> |
| <li>COMMENTS</li> |
| <li>UNIX_LINES</li> |
| </ul> |
| |
| <p> |
| These can be OR'd in the usual manner. <b>NOTE:</b> when using the |
| regex on the <a href="#MatchType"><code>match</code> </a> type, the |
| expression can contain variable references to be resolved in the |
| environment. |
| </p> |
| |
| <p> |
| <code>file-match</code> |
| exports the attributes pertaining to |
| <code>org.eclipse.core.filesystem.IFileInfo</code> |
| ; <i>efsAttributes</i> is an OR'd string of the EFS constants: |
| </p> |
| |
| <ul> |
| <li>ATTRIBUTE_READ_ONLY</li> |
| <li>ATTRIBUTE_IMMUTABLE</li> |
| <li>ATTRIBUTE_OWNER_READ</li> |
| <li>ATTRIBUTE_OWNER_WRITE</li> |
| <li>ATTRIBUTE_OWNER_EXECUTE</li> |
| <li>ATTRIBUTE_GROUP_READ</li> |
| <li>ATTRIBUTE_GROUP_WRITE</li> |
| <li>ATTRIBUTE_GROUP_EXECUTE</li> |
| <li>ATTRIBUTE_OTHER_READ</li> |
| <li>ATTRIBUTE_OTHER_WRITE</li> |
| <li>ATTRIBUTE_OTHER_EXECUTE</li> |
| <li>ATTRIBUTE_EXECUTABLE</li> |
| <li>ATTRIBUTE_ARCHIVE</li> |
| <li>ATTRIBUTE_HIDDEN</li> |
| <li>ATTRIBUTE_SYMLINK</li> |
| <li>ATTRIBUTE_LINK_TARGET</li> |
| </ul> |
| |
| <p> |
| The <i>lastModified...</i> fields require the format |
| <code>yyyy/MM/dd HH:mm:ss</code> |
| . |
| </p> |
| |
| <h4> |
| <a name="Files">2. Managed Files</a> |
| </h4> |
| |
| <p> |
| By "managed file" is meant a local file required by the executable but |
| which may not be present on the host on which the job will run; hence, |
| these files may either be external or may actually be generated from |
| the resource manager environment in conjunction with the job |
| submission, but in any case need to be copied to that host just prior |
| to it. The |
| <code>script</code> |
| file used in connection with scheduler (batch) jobs is a special |
| category of the |
| <code>managed-file</code> |
| type, and will be discussed in the next sub-section. |
| </p> |
| |
| <img alt="ManagedFiles" src="images/05managed-files.jpeg" /> |
| |
| <p> |
| Managed files are added to the definition in groups determined by |
| their shared staging location, which is a path relative to the working |
| directory of the connection. (The batch script is by default staged to |
| <i> .eclipsesettings</i> in the user home directory, since it is by |
| default temporary and deleted after the submit call returns; however, |
| one can use the |
| <code>file-staging-location</code> |
| on the |
| <code>script</code> |
| element to set the path explicitly.) The file itself can be set to be |
| deleted after submission (<i>deleteTargetAfterUse</i>; this is |
| currently not implemented in the general case) and if the file is |
| actually generated by the resource manager, the local temporary copy |
| can be deleted or retained (<i>deleteSourceAfterUse</i>). In the |
| latter case, one can also distinguish multiple copies of the generated |
| file by setting <i>uniqueIdPrefix</i> to true. |
| </p> |
| |
| <p>The managed file definition allows one of three possibilities as |
| to specifying content.</p> |
| <ol> |
| <li>If it is an external file or one edited perhaps through the |
| workspace, one can use the <code>path</code> element to indicate |
| location; this can be either a hard-coded path or a reference to a |
| property or attribute value.</li> |
| <li>The <code>contents</code> element is a string which literally |
| represents the file contents. The <i>resolveContents</i> attribute in |
| this case is used to indicate whether to pass this string through the |
| variable resolver, substituting any references to property or |
| attribute values it may contain. However, caution should be exercised |
| here in that this string should not contain "${...}" sequences which |
| do not actually refer to Eclipse variables (such as batch script |
| variables), or the resolution will fail. If <i>resolveContents</i> is |
| set to false, you can still provide this text element with a single |
| reference to a property or attribute, in which case the value of that |
| property or attribute will be used <i>as is</i>, without further |
| dereferencing.</li> |
| <li>The preferred, and most flexible way to represent the |
| contents of a file to be generated, however, is to use the <code>line</code> |
| type which is discussed <a href="#LineArg">below</a> in connection |
| with the <code>script</code> type.</li> |
| |
| </ol> |
| |
| <p>When the submit call is executed, managed files are written out, |
| if so indicated, and then their target paths are determined as |
| follows:</p> |
| |
| <ul> |
| <li>If the <code>path</code> element is used, the target path |
| becomes <i>staging directory / name of pre-existing file</i>;</li> |
| <li>If the <code>content</code> or <code>line</code> elements are |
| used, the target path becomes <i>staging directory/ |
| [uniqueIdPrefix]managed-file-name</i>);</li> |
| <li>A property is placed in the environment whose <i>name</i> is |
| the managed-file <i>name</i> and whose <i>value</i> is this target |
| path.</li> |
| </ul> |
| |
| <h4 align="center"> |
| <a name="Script">The (Batch) Script Element</a> |
| </h4> |
| |
| <p>A resource manager definition for a scheduler system such as PBS |
| or LoadLeveler can be instrumented to work without a batch script (by |
| setting command-line flags or passing all the job control information |
| as environment variables, for instance), but most frequently the use |
| of a script allows more flexibility in configuring the job |
| (interactive managers of course have no need of a script).</p> |
| |
| <img alt="ScriptType" src="images/06script.jpeg" /> |
| |
| <p> |
| If the script is specified in the XML definition, its path is |
| automatically added to the list of managed files to be staged to the |
| appropriate directory (by default <i>.eclipsesettings</i>, or |
| whereever indicated by the |
| <code>file-staging-location</code> |
| element), and so there is no need to include a script entry explicitly |
| under the |
| <code>managed-files</code> |
| element. As with the |
| <code>managed-file</code> |
| , <i>deleteAfterSubmit</i> indicates that the script target should not |
| be retained (this is the default behavior); unlike the |
| <code>managed-file</code> |
| , however, the local copy of the generated script is always deleted. A |
| reserved property, |
| <code>managed_file_for_script</code> |
| , should be used to reference the script's path on the target |
| resource; for instance, in the PBS submit command: |
| </p> |
| |
| <code> |
| <arg>qsub</arg><br> |
| <arg>${ptp_rm:managed_file_for_script#value}</arg> |
| </code> |
| <br> |
| |
| <p> |
| <b>Note</b>: If the |
| <code>import</code> |
| tab (see <a href="#LaunchTab">below</a>) is used to provide an |
| external or workspace edited batch script to the run, nothing extra |
| need be done in the definition XML, as the presence of any external |
| path for the script is handled under the covers. |
| </p> |
| |
| <p> |
| <i>insertEnvironmentAfter</i> is a line number indicating where in the |
| script to add any extra environment variables set through the <b>Environment</b> |
| tab provided as part of the <b>Run Configuration</b> wizard. This way |
| the user has control over whether these should overwrite previously |
| defined environment variable values. |
| </p> |
| |
| <p></p> |
| |
| <h4 align="center"> |
| <a name="LineArg"><code>line</code> and <code>arg</code> types</a> |
| </h4> |
| |
| <p> |
| The |
| <code>line</code> |
| element was mentioned <a href="#Files">above</a> in connection with |
| managed file content; it is essentially a wrapper around a series of |
| whitespace separated |
| <code>arg</code> |
| elements, all of which are placed on a single line (that is, the group |
| is terminated by a line separator). |
| </p> |
| |
| <img alt="Line" src="images/07line-arg.jpeg" /> |
| |
| <p> |
| The |
| <code>arg</code> |
| element is used for script and managed file content as well as in the |
| definition of commands. Its text element can contain variable |
| references to be resolved against the environment, but also allows for |
| the entire string to be treated as a literal by setting <i>resolve</i> |
| to false (default is true). This is useful inasmuch as it allows for |
| the presence of batch-type variables (e.g., ${HOME}) which should be |
| resolved by the remote shell and not inside the Eclipse client. |
| </p> |
| <p> |
| The default behavior of the argument resolver is not to write out or |
| include arguments whose value is undefined (either |
| <code>null</code> |
| or zero-length). <i>isUndefinedIfMatches</i> affords more nuanced |
| control over whether an argument should be so eliminated. If the |
| argument references property or attribute values, but also has |
| text-literal segments, a regex can be provided to define what an |
| "empty" argument would be in this case. For instance, if an argument |
| flag should not appear when the value it precedes is an empty string, |
| one could write: |
| </p> |
| |
| <code> <arg isUndefinedIfMatches="-f">-f |
| ${ptp_rm:flag#value}</arg> </code> |
| <br> |
| |
| <p>For the purposes of matching, trailing whitespace is trimmed |
| from the resolved argument, so there is no need to specify this as |
| part of the regex used to match.</p> |
| |
| <h4 align="center"> |
| <a name="JustInTimeEval">"Just-in-time" resolution of <i>@jobId</i> |
| and <code>managed-file</code> paths</a> |
| </h4> |
| |
| <p> |
| <i>@jobId</i> is a special property name designating the runtime id |
| for a job instance. In the lifecycle of the run/launch (submit) call, |
| this value begins as an internally generated unique id which then is |
| swapped for the id returned by the scheduler or runtime system. |
| </p> |
| |
| <p> |
| The <i>@jobId</i>, along with the target paths for |
| <code>managed-file</code> |
| elements, are not known at configuration time (i.e., before the user |
| hits "Run"). While the former is made visible to the parsers and the |
| returned status object of the submit command, neither is in the scope |
| of (available for reference in) other managed files or the |
| <code>script</code> |
| element, because these latter elements are generated just prior to the |
| actual submission. |
| </p> |
| |
| <p> |
| If the |
| <code>script</code> |
| needs to refer to the <i>@jobId</i>, it must do so via the variable |
| made available by the particular scheduler it is written for. An |
| example of how to reference the target path of a |
| <code>managed-file</code> |
| inside the |
| <code>script</code> |
| is included in the <a href="http://download.eclipse.org/tools/ptp/docs/JAXBDemo.pdf">tutorial slides</a>; this |
| essentially involves defining a variable in the submission command's |
| environment, with a reference to the |
| <code>managed-file</code> |
| path property as its value, and then using this environment variable |
| inside the |
| <code>script</code> |
| . |
| </p> |
| |
| <h4> |
| <a name="CommandsParsers">3. Commands</a> |
| </h4> |
| |
| <p> |
| As mentioned <a href="#CommandTypes">above</a>, the resource manager |
| <code>command</code> |
| denotes a [UNIX-type] system call made on the resource manager's |
| remote (target) connection (PTP does not generally support execution |
| on Windows systems). The command is always configured and executed |
| through a Java API (the "process builder") which bottoms out in a |
| <code>bash -c</code> |
| command. The first argument of the command thus must be the name/path |
| of the executable only. Its own arguments can then be subsequently |
| specified by an arbitrary number of |
| <code>arg</code> |
| elements. |
| </p> |
| |
| <img alt="CommandType" src="images/08command.jpeg" /> |
| |
| <p> |
| There are a number of important attributes on the |
| <code>command-type</code> |
| element which bear detailed discussion here. <i>directory</i> of |
| course refers to where the command will be executed; <b>this |
| defaults to the "home" or working directory of the remote control |
| connection (the <code>control.working.dir</code> property)</b>; also as |
| expected, <i>redirectStderr</i> means that both output and error |
| streams are sent back on |
| <code>stdout</code> |
| . One can, if necessary, tune the buffer sizes on the stream readers |
| via the <i>streamBufferLimit</i>. If the environment set on the |
| command should entirely replace the shell environment (the default |
| behavior is to append the command environment), set <i>replaceEnvironment</i> |
| to true. |
| </p> |
| |
| <p> |
| If you do not wish an error to be thrown in the case of non-zero exit |
| of the command, set <i>ignoreExitStatus</i> to true. The <i>flags</i> |
| attribute is an OR'd string of three possible values: |
| </p> |
| |
| <ul> |
| <li>NONE (default)</li> |
| <li>ALLOCATE_PTY (allocates a pseudo-terminal)</li> |
| <li>FORWARD_X11</li> |
| </ul> |
| |
| <p> |
| Two crucial fields for command control are <i>waitForId</i> and <i>keepOpen</i>. |
| The former indicates to the resource manager that the output stream |
| for the command is being parsed for an id which will appear as a |
| property in the environment during the command execution, and that the |
| execution should not return until it sees this id. Very often the |
| submit commands will have these semantics. The latter field indicates |
| that the command should be held open for potentially repeated |
| redirection of input (see next paragraph). There can only be one such |
| process open inside the resource manager at a time (these semantics |
| will usually pertain to interactive managers using a pseudo-terminal |
| mode to control job submission). |
| </p> |
| |
| <hr> |
| <h5> |
| <a name="Note2">BEST PRACTICE: WAIT-FOR-ID</a> |
| </h5> |
| <p> |
| As <a href="#CommandTypes">previously</a> described, there are two |
| basic types of commands, <b>(uu)id</b> commands which are related to a |
| specific job, and generic (non-uuid) commands. |
| <code>submit-</code> |
| and |
| <code>-job-</code> |
| commands fall into the former category; those in the latter category |
| are usually simple interactive commands, and their output is not |
| redirected to the console, but uuid-commands have several subclasses: |
| </p> |
| <br> |
| <table align="CENTER" rules="all"> |
| <tr> |
| <th>COMMAND TYPE</th> |
| <th>WAIT FOR ID</th> |
| <th>TOKENIZER STATES</th> |
| </tr> |
| <tr> |
| <td>batch</td> |
| <td>yes</td> |
| <td>SUBMITTED</td> |
| </tr> |
| <tr> |
| <td>interactive</td> |
| <td>no</td> |
| <td></td> |
| </tr> |
| <tr> |
| <td>interactive</td> |
| <td>yes</td> |
| <td>RUNNING</td> |
| </tr> |
| <tr> |
| <td>interactive, open</td> |
| <td>no</td> |
| <td></td> |
| </tr> |
| <tr> |
| <td>interactive, open</td> |
| <td>yes</td> |
| <td>(SUBMITTED,) RUNNING</td> |
| </tr> |
| <tr> |
| <td>interactive, open, pseudo-terminal</td> |
| <td>no</td> |
| <td></td> |
| </tr> |
| <tr> |
| <td>interactive, open, pseudo-terminal</td> |
| <td>yes</td> |
| <td>(SUBMITTED,) RUNNING</td> |
| </tr> |
| </table> |
| <br> |
| <p> |
| This table shows the various combinations that are feasible between |
| command type and waiting states. Simple interactive commands, whether |
| held open for further input or not, can wait or not wait, but in the |
| case of waiting, they should be provided with a <a |
| href="#StreamParsers">stream tokenizer</a> which recognizes and sets |
| the RUNNING state. Batch jobs, on the other hand, will usually have a |
| tokenizer which recognizes and sets SUBMITTED. In the case of some |
| open-process jobs (e.g., |
| <code>qsub -I</code> |
| for PBS, which also requires a pseudo-terminal to connect remotely), |
| there will usually be a pause before the terminal job is actually |
| scheduled; in order to allow the user to see that the job has been |
| accepted and is pending, the tokenizer needs to set both SUBMITTED and |
| RUNNING states, the latter when the job has actually started. |
| </p> |
| <hr> |
| |
| <p> |
| We have discussed the |
| <code>arg</code> |
| type <a href="#LineArg">above</a>. Note that the command arguments, |
| input and even environment variables make use of this element. For the |
| <code>name-value-pair</code> |
| type comprising the latter one can as simple alternative set the <i>value</i> |
| attribute to a string (which will be resolved first in the current |
| environment); finer-grained control over the resolution of the value, |
| however, requires the use of the |
| <code>arg</code> |
| type. When there is |
| <code>input</code> |
| present, it is directed to the input stream of the command. If the <i>keepOpen</i> |
| attribute is true, a check will be made to see if there already exists |
| an open process (which is also alive), which will then be used; |
| otherwise, the arguments are executed, then the input arguments are |
| given to the process. With an open command/process, the input |
| arguments can be fed repeatedly to the same process; this allows, for |
| instance, for continuous testing of an interactive job in the same |
| interactive session. |
| </p> |
| |
| <h4 align="center"> |
| <a name="StreamParsers">Stream Parsers</a> |
| </h4> |
| |
| <p> |
| One can attach parsers (which we also refer to as tokenizers) to the |
| output and error streams of any |
| <code>command</code> |
| in order to capture information and use it to side-effect existing |
| properties or attributes, or to generate new ones on the fly. While |
| the parser is not completely general, it is capable of a wide range of |
| tasks which would typically be required in the handling of output from |
| batch and runtime systems. We here explain the main elements ( |
| <code>target, match, test</code> |
| ) used by the tokenizer; one should further consult the <a |
| href="#TokenizerExamples">tokenizer examples</a> demonstrating |
| various usage scenarios. |
| </p> |
| |
| <img alt="TokenizerType" src="images/090tokenizer.jpeg" /> |
| |
| <p> |
| The |
| <code>type</code> |
| element will most commonly not be set, meaning the built-in parser |
| will be used; however, it is possible to implement a custom parser as |
| a contribution to the |
| <code>org.eclipse.ptp.rm.jaxb.core.streamParserTokenizer</code> |
| extension point, in which case this element should be set to its |
| extension id. Note that the extension requires the class to implement |
| <code>org.eclipse.ptp.rm.jaxb.control.internal.IStreamParserTokenizer</code> |
| , which is a Runnable interface with an initialize method which passes |
| in any job id plus the current environment map; the details of such a |
| parser's implementation are not, however, configured from the XML |
| document. |
| </p> |
| |
| <p> |
| The built-in tokenizer can read the stream in two different ways. If <i>delim</i> |
| is provided, the stream is split using the indicated value. The string |
| should only be one character in length (escaped or non-escaped). |
| Provision is made for the '\r\n' (Windows) two-character delimiter |
| internally; in this case the delimiter should be set to "\r" (however, |
| as already mentioned, PTP does not generally guarantee that system |
| calls will work on Windows). Setting <i>includeDelim</i> means that |
| the delimiter will appear as the last char on the returned stream |
| segment. |
| </p> |
| |
| <p> |
| The second way to read from the stream is to provide a <i>maxMatchLen</i> |
| size; what this indicates is that whatever substring needs to be found |
| on the stream will not exceed this length. The stream is then read in |
| swatches of <i>maxMatchLen</i>, with the internal buffer set to twice |
| this size, so that each successive read shifts the buffer to the |
| "left" by one length. This guarantees that all such substrings will |
| eventually be matched. |
| </p> |
| |
| <p> |
| Sometimes a sort of "look-ahead" paradigm is necessary. For instance, |
| one may need to match a segment or segments whose position is defined |
| from the end of the output, but you do not know in advance the actual |
| stream length. In this case, one can opt to read until the end of the |
| stream (<i>all</i>="true"), retaining only the last <i>N</i> |
| buffer-lengths or delimited segments, as indicated by the <i>save</i> |
| field. When the parser reaches the end of the stream, it will then |
| apply the various targets to each saved segment in order. |
| </p> |
| |
| <p> |
| <i>applyToAll</i> is discussed further under |
| <code>target</code> |
| . The |
| <code>exit-on</code> |
| element indicates that the tokenizer should quit immediately when it |
| encounters this pattern; |
| <code>exit-after</code> |
| indicates that the tokenizer should quit when it encounters this |
| pattern, but should first apply the current segment to its targets. |
| </p> |
| |
| <h5 align="CENTER"> |
| <a name="TargetType">TARGET</a> |
| </h5> |
| |
| <img alt="TargetType" src="images/091target.jpeg" /> |
| |
| <p> |
| A tokenizer may be given any number of |
| <code>target</code> |
| elements. The target denotes a particular value (object) currently in, |
| or to be written to, the environment, which will be side-effected on |
| the basis of the result of the tokenization. A target in turn contains |
| <code>match</code> |
| elements and |
| <code>test</code> |
| elements; the former are run as part of the stream processing; the |
| latter are run after the stream processing has been completed. The |
| optional |
| <code>else</code> |
| element is applied only if there are no other tests defined or if none |
| of the defined tests succeed). |
| </p> |
| |
| <p> |
| The target object is either to be constructed at match time, or it |
| pre-exists in the environment. If constructed, the <i>type</i> field |
| is used to indicate whether the object is a |
| <code>property</code> |
| or an |
| <code>attribute</code> |
| type; otherwise, <i>ref</i> points to the name of the |
| <code>property</code> |
| or |
| <code>attribute</code> |
| in the environment (recall that for the runtime job identifier, <i>@jobId</i> |
| is used). |
| </p> |
| <p> |
| <b>Note</b>: when new targets are constructed, there is a merge |
| operation at the end of tokenization which attempts to combine objects |
| into a single instance identified by their <i>name</i> attribute. This |
| assumes that such names will be unique and that any other values to be |
| set on the object which are not explicitly bound in some way to that |
| name via the match pattern will appear on the stream before a new name |
| does (see ex. 5 in <a href="#TokenizerExamples">tokenizer examples</a>). |
| <b><i>The default behavior of this merge is that it will fail |
| if two objects with the same name but differing values are generated |
| by the parsing.</i> (This excludes <code>add</code> and <code>put</code> |
| operations which create a list or map; in these cases, the two |
| collections or maps will be combined into one. This does <i>NOT</i> |
| work, however, for <code>append</code> actions.) To allow duplicates, |
| set the <i>allowOverwrites</i> to true; in this case, successive |
| duplicates simply replace the preceding object. </b> |
| </p> |
| |
| <p>The default behavior of the tokenizer read-match sequence is as |
| follows:</p> |
| |
| <ol> |
| <li>read from the stream either a set number of chars or until |
| the delimiter is found;</li> |
| <li>for each target: |
| <ul> |
| <li>for each match: |
| <ul> |
| <li>if the match is positive, process each of the actions, |
| and go to (1).</li> |
| </ul> |
| </ul></li> |
| </ol> |
| |
| <p> |
| Only one qualifying target is processed for any given segment read, |
| and for the given target, the first pattern matched is the one |
| processed for its actions. This is basically the "OR" semantics of |
| normal logic programming; hence <i>the implementer must be careful |
| to arrange the matches inside a target in such a way that the more |
| specific match patterns precede the more general</i>. |
| </p> |
| <p>Three boolean fields allow you to modify this behavior.</p> |
| |
| <ol> |
| <li>The <i>applyToAll</i> field on the <code>tokenizer-type</code> |
| element means take the unmatched part of the read stream and pass it |
| to the next target, even if there was a previous match; this allows |
| you to capture more than one regex pattern per stream segment (see |
| ex. 6 in <a href="#TokenizerExamples">tokenizer examples</a>).</li> |
| <li>The <i>matchAll</i> field on the <code>target-type</code> |
| element means do not try to match an already matched expression until |
| all the others are matched (i.e., a logical AND instead of OR governs |
| the set of matches at successive calls to the target match |
| operation); this allows one to use, for instance, .* repeatedly but |
| set different fields of the object with the resulting match (see ex. |
| 5 in <a href="#TokenizerExamples">tokenizer examples</a>).</li> |
| <li>The <i>moveToTop</i> field on the <code>match-type</code> |
| element indicates to the tokenizer that the matched target be |
| promoted to first position in the list of targets. This is useful |
| when there is an ordering which expects types of attributes or |
| properties to be grouped in sequence on the stream (see ex. 4 in <a |
| href="#TokenizerExamples">tokenizer examples</a>).</li> |
| </ol> |
| |
| <p>When a match is found, the set of action types it contains are |
| all applied.</p> |
| |
| <h5 align="CENTER"> |
| <a name="MatchType">MATCH</a> |
| </h5> |
| |
| <img alt="MatchType" src="images/10match.jpeg" /> |
| |
| <p>Each of these types corresponds to an action to be taken on the |
| indicated field of the target object.</p> |
| |
| <table align="center" rules="all"> |
| <tr> |
| <td><code>set</code></td> |
| <td>sets the value of that field</td> |
| </tr> |
| <tr> |
| <td><code>append</code></td> |
| <td>adds to a string buffer, whose string value will be set on |
| the field</td> |
| </tr> |
| <tr> |
| <td><code>add</code></td> |
| <td>adds to a list to which the value of that field will be set</td> |
| </tr> |
| <tr> |
| <td><code>put</code></td> |
| <td>places a key-value pair in a map to which the value of that |
| field will be set</td> |
| </tr> |
| <tr> |
| <td><code>throw</code></td> |
| <td>throws an exception and (optionally) also sets the value of |
| the field</td> |
| </tr> |
| </table> |
| |
| <p> |
| The actions listed here all have |
| <code>entry-type</code> |
| children, either single ( |
| <code>set, throw</code> |
| ) or potentially multiple. All of these except |
| <code>throw</code> |
| also allow you to force the creation of a new object (<i>forceNewObject</i>) |
| each time it is applied; the new object then replaces the current one |
| for successive actions in the match. |
| </p> |
| |
| <h5 align="CENTER"> |
| <a name="EntryType">ENTRY</a> |
| </h5> |
| |
| <img alt="EntryType" src="images/11entry.jpeg" /> |
| |
| <p> |
| This value-abstraction allows one to set <i>key</i> (for maps) and <i>value</i> |
| as literals or references to other properties or attributes to be |
| resolved in the current environment; to reference the matched segment |
| parts one sets <i>keyIndex</i> and <i>valueIndex</i> if the regex was |
| used to split the segment; otherwise, <i>keyGroup</i> and <i>valueGroup</i> |
| refer to the capture group of the regex pattern, with group 0 |
| referring to the entire match. |
| </p> |
| |
| <h5 align="CENTER"> |
| <a name="TestType">TEST</a> |
| </h5> |
| |
| <img alt="TestType" src="images/12test.jpeg" /> |
| |
| <p> |
| As mentioned <a href="#TargetType">above</a>, the |
| <code>test-type</code> |
| elements are all run after the tokenization has reached the end of the |
| stream. This class of actions is useful for setting values based on |
| other values produced during tokenization. A test is one or more |
| comparison operations plus a set of actions to apply to the target |
| fields in the case of either success or failure (the "else" element); |
| see ex. 3 or the "get-job-status" example in <a |
| href="#TokenizerExamples">tokenizer examples</a>. |
| </p> |
| |
| <p> |
| The <i>op</i> attribute can be one of the following comparisons: |
| </p> |
| |
| <table align="center"> |
| <tr> |
| <td>EQ</td> |
| <td>:</td> |
| <td>equals</td> |
| </tr> |
| <tr> |
| <td>LT</td> |
| <td>:</td> |
| <td>less than</td> |
| </tr> |
| <tr> |
| <td>LE</td> |
| <td>:</td> |
| <td>less than or equal to</td> |
| </tr> |
| <tr> |
| <td>GT</td> |
| <td>:</td> |
| <td>greater than</td> |
| </tr> |
| <tr> |
| <td>GE</td> |
| <td>:</td> |
| <td>greater than or equal to</td> |
| </tr> |
| </table> |
| |
| <p> |
| When the operation is set to one of these, it is expected that the two |
| <code>value</code> |
| elements will be used. As usual, these elements can be literals or can |
| contain variables to be resolved into a string type; |
| <code>#FIELD</code> |
| refers to the value of the given field on the current target; the |
| strings will be converted in conformity with the inferred (primitive) |
| type of the comparison. The |
| <code>else</code> |
| element also pertains to comparison tests; the actions listed there |
| will be taken upon failure of the comparison. |
| </p> |
| |
| <p> |
| The <i>op</i> attribute can also be a logical operator [AND, OR, NOT], |
| in which case the embedded |
| <code>test</code> |
| object should be used; these can be nested to an arbitrary depth, but |
| of course must bottom out in a comparison operation. |
| </p> |
| |
| <h4 align="center"> |
| <a name="TokenizerExamples"></a><b>Contents of Tokenizer Examples |
| (</b><a href="tokenizer-examples.xml">tokenizer-examples.xml</a>) |
| </h4> |
| <br> |
| |
| <table align="center" rules="all"> |
| <tr> |
| <th>Example</th> |
| <th>Description</th> |
| </tr> |
| <tr> |
| <td>1</td> |
| <td>output is a list of line-separated queue names to be |
| assigned to the known property "available-queues"</td> |
| </tr> |
| <tr> |
| <td>2</td> |
| <td>output is to be searched for its final line which should |
| contain a job id of the form "[digits].[chars]"</td> |
| </tr> |
| <tr> |
| <td>3</td> |
| <td>indeterminate number and order of lines containing parts of |
| attribute definitions, but each line bearing a distinct id (e.g., |
| openMPI attribute discovery)</td> |
| </tr> |
| <tr> |
| <td>4</td> |
| <td>indeterminate number of definitions, but grouped by caption; |
| use of <i>moveToTop</i> to promote the target to the top of the list |
| when the caption appears</td> |
| </tr> |
| <tr> |
| <td>5</td> |
| <td>similar to 4, but without delimiter (implicit ordering)</td> |
| </tr> |
| <tr> |
| <td>6</td> |
| <td>similar to 4, but with indeterminate type order and using |
| buffer + DOTALL | UNIX_LINES</td> |
| </tr> |
| <tr> |
| <td>7</td> |
| <td>indeterminate number of property definitions, but on single |
| line</td> |
| </tr> |
| <tr> |
| <td>8</td> |
| <td>looking for values interspersed in the stream but which will |
| not exceed 32 chars</td> |
| </tr> |
| <tr> |
| <td>9</td> |
| <td>successive names/values, in order, but staggered</td> |
| </tr> |
| <tr> |
| <td>10</td> |
| <td>forced merge</td> |
| </tr> |
| <tr> |
| <td>11</td> |
| <td>exit-on</td> |
| </tr> |
| <tr> |
| <td>12</td> |
| <td>exit-after</td> |
| </tr> |
| <tr> |
| <td>13</td> |
| <td>get-job-status (use of tests)</td> |
| </tr> |
| </table> |
| <br> |
| |
| <h4> |
| <a name="LaunchTab">4. The Launch Tab</a> |
| </h4> |
| |
| <p> |
| The UI component which needs to be configured in conjunction with the |
| resource manager control is the <b>Launch Tab</b>, which is actually |
| the main area of the <b>Resources Tab</b>, one of the six tabs |
| comprising the <b>Run Configuration</b> wizard or the seven tabs |
| comprising the <b>Debug Configuration</b> wizard. The purpose of this |
| component is to allow the user to set up the job by providing values |
| relating to resource requirements or to the properties of the job in |
| general. In other words, it is in this component that the user can set |
| the values on the |
| <code>attribute</code> |
| elements making up the configurable environment as defined in the XML. |
| </p> |
| |
| <p> |
| Specifying the composition and layout of this component can be tricky |
| and tedious because it deals with widgets defined by the Eclipse |
| <code>JFace</code> |
| library and with their underlying implementation in |
| <code>SWT</code> |
| . We intend in the near future to provide some standard "templates" |
| for typical widget groups that can be cut and pasted into the XML |
| definition. Some of this can already be achieved by importing into the |
| workspace a provided definition (such as for the PBS resource manager) |
| and modifying it or borrowing from it. This is, in fact, the approach |
| demostrated in the <a href="http://download.eclipse.org/tools/ptp/docs/JAXBDemo.pdf">tutorial</a>. |
| </p> |
| |
| <p> |
| <b> Throughout the following, it may generally be assumed that |
| fields such as <i>style</i>, <i>foreground</i>, <i>background</i>, |
| etc., take string equivalents of the corresponding <code>SWT</code> |
| constants (e.g., "SWT.LEFT", "SWT.VERTICAL", "SWT.NONE") which can be |
| OR'd (using "|") wherever they would be in Java code. We will <i>not</i> |
| exhaustively specify here which constants pertain to which widgets, |
| because in most cases we have tried to reflect very closely the names |
| of the <code>SWT</code> object types and fields, and this information |
| can be found by consulting the Javadocs relative to the widgets in |
| question. </b> |
| </p> |
| |
| <img alt="LaunchTabType" src="images/13launch-tab.jpeg" /> |
| |
| <p> |
| As seen from this top-level specification, the Launch Tab can have any |
| number of |
| <code>dynamic</code> |
| tab-controllers, "dynamic" meaning configurable from the XML, which |
| will discuss in more detail in the following paragraphs. |
| </p> |
| <p> |
| There is also a fixed |
| <code>import</code> |
| controller type (implemented as an extension/subclass of the dynamic |
| controller, in order to provide uniform handling of the environment); |
| this component allows you to browse the workspace to set a custom |
| script as the one to run. Its main text area widget is read-only; any |
| editing of the script must be done through the workspace editor. The |
| browse is local only, so remote scripts should first be imported into |
| a local project in the workspace (using the remote systems import |
| wizard), and then captured by this controller's browse button. |
| </p> |
| |
| <p> |
| The configurable part of the import tab is an optional <a |
| href="#AttributeViewer">attribute viewer</a> for selecting the |
| variables from the environment which you wish to pass to the launch |
| command along with the script. <i>The values set here should |
| correspond to whatever the script contains; they are set for the |
| benefit of the internal consistency of the PTP client, and do not |
| override those hard-coded in script.</i> For an example, see the <a |
| href="PLUGINS_ROOT/org.eclipse.ptp.rm.jaxb.pbs.doc.user/html/PBSHelpPage.html#ImportTab">Import |
| PBS Script</a> tab. |
| </p> |
| |
| <h4 align="CENTER"> |
| <a name="TabController">Tab Controller</a> |
| </h4> |
| |
| <p> |
| The |
| <code>dynamic</code> |
| controllers belong to the |
| <code>tab-controller-type</code> |
| , which contains an arbitrary number of |
| <code>tab-folder,</code> |
| <code>composite,</code> |
| <code>widget,</code> |
| <code>browse,</code> |
| <code>button-group,</code> |
| <code>action,</code> |
| or |
| <code>viewer</code> |
| elements; as will be seen shortly, the first two, which are |
| "containers", are potentially recursive elements, so that one can nest |
| widgets in the usual manner. For all intents and purposes, the |
| <code>tab-controller-type</code> |
| is the same as a |
| <code>composite</code> |
| , but without the <i>group</i> attribute and with the extra field for |
| controlling shared widgets. |
| </p> |
| |
| <img alt="TabController" src="images/14tab-controller.jpeg" /> |
| <p> |
| The <i>includeWidgetValuesFrom</i> field is important. By default, it |
| is set to empty, meaning that the subset of the total resource manager |
| environment represented by the controller's widgets is all that will |
| be passed to the actual job configuration when the job is launched |
| from this tab. This behavior can be changed by giving a |
| comma-delimited list of the |
| <code>title</code> |
| values for other controllers. In this case, the configuration for the |
| job, when the "Run" button is pressed with this controller being the |
| visible one, will also include <i>all the properties or attributes |
| valid for the listed but currently invisible controllers</i>. |
| </p> |
| <hr> |
| <h5> |
| <a name="Note3">PROPERTY/ATTRIBUTE NAMES vs VALUES</a> |
| </h5> |
| |
| <p>While this field controls the inclusion or exclusion of |
| attribute and property names in the configuration, their values are |
| always shared across the controllers (there is only one environment); |
| thus if a variable value is changed on one controller, and this |
| variable is also exposed on another, the value for that variable will |
| also change on the other controller when it becomes visible; |
| similarly, if the user switches off a controller to another having |
| only a subset of the first controller's variables, the variables not |
| represented on the second controller will become undefined or return |
| to their defaults.</p> |
| <p> |
| <b>Note</b> that the union of valid names is not transitively closed; |
| that is, if Tab1 has <i>includeWidgetValuesFrom</i>="Tab2", and Tab2 |
| similarly has <i>includeWidgetValuesFrom</i>="Tab3", Tab1 will get |
| only the valid values local to Tab2. <b>Note</b> also, that if there |
| are names which intersect with strictly local widget targets, the |
| local settings (visibility, selection, enablement) still determine |
| whether they belong in the set of valid property/attribute names. |
| </p> |
| <hr> |
| <p> |
| The <i>showViewConfig</i> and <i>showViewExcluded</i> fields allow |
| control over whether the "View Configuration" and "View Excluded" |
| buttons appear at the bottom of the tab (the "View Script" button |
| appears if the configuration contains a |
| <code>script</code> |
| element). <i>showViewConfig</i> is true by default; this button brings |
| up a scrolling dialog which allows you to inspect the current |
| configuration (that is, the one relevant to the visible controller). <i>showViewExcluded</i> |
| is by default false, and should only be set if the resource manager |
| processes attributes or properties discovered by a command at run |
| time; this dialog allows you to see which ones have been marked |
| invisible; <b>unlike invisible predefined properties or attributes</b>, |
| which are always <i>included</i>, invisible discovered properties or |
| attributes are always <i>excluded</i> from the configuration and thus |
| do not persist across resource manager sessions. |
| </p> |
| <br> |
| <h4 align="CENTER"> |
| <a name="TabFolders">Tab Folders and Tab Items</a> |
| </h4> |
| |
| <img alt="TabFolder" src="images/15tab-folder.jpeg" /> |
| <p> |
| The |
| <code>tab-folder</code> |
| is a container which looks like a file-folder, with tabs running |
| across the top or bottom (<i>style</i> includes SWT.TOP, SWT.BOTTOM). |
| Each tab label corresponds to a |
| <code>tab-item</code> |
| which provides the actual visible content. The |
| <code>tooltip</code> |
| pertains to the entire folder. |
| </p> |
| |
| <p> |
| The |
| <code>tab-item</code> |
| is another all-purpose "container" in which all widget types can be |
| embedded, but distinguished from the <a href="#Composite"><code>composite</code> |
| </a> in that it can only be the child of a |
| <code>tab-folder</code> |
| and that it can have a tooltip (which appears on the tab). |
| </p> |
| |
| <p> |
| As with |
| <code>composite</code> |
| (see <a href="#Composite">below</a>), the tab folder and tab item take |
| <code>layout</code> |
| and |
| <code>layout-data</code> |
| elements; these closely reflect their |
| <code>SWT</code> |
| objects and types; for further explanation we refer the reader once |
| again to the relevant Javadocs. |
| </p> |
| |
| <h4 align="CENTER"> |
| <a name="Layout">Layout</a> |
| </h4> |
| <img alt="Layout" src="images/17layout.jpeg" /> |
| <h4 align="CENTER"> |
| <a name="LayoutData">Layout Data</a> |
| </h4> |
| <img alt="LayoutData" src="images/18layout-data.jpeg" /> |
| |
| <h4 align="CENTER"> |
| <a name="Composite">Composite/Group</a> |
| </h4> |
| |
| <p> |
| The main container, of course, is the |
| <code>composite</code> |
| ; its sub-type, the group (<i>group</i>="true"), allows for decoration |
| and a title, while the simple composite acts as a widget holder and |
| has no visible characteristics, though it does have a layout. As is |
| evident, composites can be contained in composites and tab-items, and |
| tab-folders can be contained in composites. |
| </p> |
| |
| <img alt="Composite" src="images/16composite.jpeg" /> |
| <br> |
| |
| <h4 align="CENTER"> |
| <a name="ControlState">Control State Wiring</a> |
| </h4> |
| |
| <p> |
| It should be evident that all the XML elements corresponding to UI SWT |
| <code>control</code> |
| types have been given |
| <code>control-state</code> |
| children. |
| </p> |
| |
| <img alt="Composite" src="images/22control-state.jpeg" /> |
| <br> |
| |
| <p> |
| The purpose of these elements is to provide an optional way of setting |
| the <b>enabled</b> and/or <b>visible</b> states of the control on the |
| basis of the state of button (checkbox/radio) selection. One can |
| define two such elements per control, using one of the show/hide or |
| enable/disable pairs. Each of these is associated with a |
| <code>control-state-rule:</code> |
| </p> |
| |
| <img alt="Composite" src="images/23control-state-rule.jpeg" /> |
| <br> |
| |
| <p>The basic rule is defined by its attributes:</p> |
| |
| <ol> |
| <li>the id of the button control to check;</li> |
| <li>whether its selected state should be true or false.</li> |
| </ol> |
| |
| <p>Rules can be nested to an arbitrary depth to form clauses |
| governed by the logical operators.</p> |
| |
| <p> |
| A |
| <code>SelectionListener</code> |
| is created for this target which subscribes to all the controls in the |
| rule; when it receives an event, it evaluates the rule, and if the |
| result is true, takes the associated action on this target. |
| </p> |
| |
| <p> |
| When one wishes to wire two or more such controls in this way, the |
| "trigger" button control must have an id unique to the resource |
| manager configuration. This will be assigned via the <i>buttonId</i> |
| element on the |
| <code>widget</code> |
| corresponding to the button (true also for the buttons nested in a |
| <code>button-group</code> |
| ). |
| </p> |
| |
| <p>Here is an example which sets the enabled and visible properties |
| of both a text field and a related checkbox:</p> |
| |
| <pre> |
| <widget type="checkbox" buttonId="002" title="Host list:" saveValueTo="checked002"> |
| <layout-data> |
| <grid-data horizontalAlign="SWT.LEFT" verticalAlign="SWT.TOP"/> |
| </layout-data> |
| <control-state> |
| <show-if button="001" selected="true"/> |
| </control-state> |
| </widget> |
| <widget type="text" style="SWT.V_SCROLL | SWT.BORDER" saveValueTo="host_list"> |
| <layout-data> |
| <grid-data horizontalAlign="SWT.FILL" verticalAlign="SWT.FILL" horizontalSpan="2" grabExcessHorizontal="true" |
| grabExcessVertical="true"/> |
| </layout-data> |
| <control-state> |
| <show-if button="001" selected="true"/> |
| <enable-if> |
| <and> |
| <rule button="001" selected="true"/> |
| <rule button="002" selected="true"/> |
| </and> |
| </enable-if> |
| </control-state> |
| </widget> |
| </pre> |
| |
| <p>Note also that this implicit dependency graph is checked for |
| cycles; the building of the UI tab will be aborted if such a cycle is |
| discovered.</p> |
| |
| <hr> |
| <h5> |
| <a name="Note4">BEST PRACTICE: CONTROL STATE</a> |
| </h5> |
| <p> |
| An invisible property should be assigned to each checkbox which |
| controls the state of other widgets (e.g., <i>checked002</i> in the |
| example above); this will allow you to restore the state of the tab |
| correctly when it is rebuilt or re-initialized. |
| </p> |
| <br> |
| <hr> |
| <h4 align="CENTER"> |
| <a name="Widget">Widget</a> |
| </h4> |
| |
| <p> |
| We now come to the five non-container widget elements with functional |
| characteristics. The first of these is the |
| <code>widget</code> |
| , which comprises a number of simple widget types. |
| <p> |
| <p> |
| The <i>type</i> of the widget can be one of the following: |
| </p> |
| |
| <br> |
| <table align="center" rules="all"> |
| <tr> |
| <td><code>label</code></td> |
| <td>fixed text for display</td> |
| </tr> |
| <tr> |
| <td><code>text</code></td> |
| <td>area for entering/editing text</td> |
| </tr> |
| <tr> |
| <td><code>checkbox</code></td> |
| <td>corresponds to a boolean choice</td> |
| </tr> |
| <tr> |
| <td><code>radiobutton</code></td> |
| <td>usually used in connection with <code>button-group</code></td> |
| </tr> |
| <tr> |
| <td><code>spinner</code></td> |
| <td>provides a continuous range of integer values</td> |
| </tr> |
| <tr> |
| <td><code>combo</code></td> |
| <td>select from a list of items</td> |
| </tr> |
| </table> |
| <br> |
| |
| <p> |
| Not all attributes and elements of the |
| <code>widget</code> |
| descriptor are applicable to all types. |
| </p> |
| <br> |
| <table align="center" rules="all"> |
| <tr> |
| <td><i>title</i></td> |
| <td>used with <code>checkbox</code> and <code>radiobutton</code> |
| widgets for the button label.</td> |
| </tr> |
| <tr> |
| <td><i>style</i></td> |
| <td>carries the <code>SWT</code> style constants appropriate to |
| the widget type.</td> |
| </tr> |
| <tr> |
| <td><i>readOnly</i></td> |
| <td>applies to <code>text</code>, <code>spinner</code> and <code>combo</code> |
| widgets; the text area in this case is not editable.</td> |
| </tr> |
| <tr> |
| <td><i>saveValueTo</i></td> |
| <td>applies to all widgets except <code>label</code>; the name |
| of the property or attribute whose value should be set on the basis |
| of the widget's "selected" value or text.<br> <b>Note:</b> use |
| of this attribute is mutually exclusive with the <code>fixed-</code> |
| and <code>dynamic-text</code> elements.</td> |
| </tr> |
| <tr> |
| <td><code>items-from</code></td> |
| <td>applies to the <code>combo</code> widget; indicates a |
| property or attribute <i>value</i> (of type <code>java.util.Collection</code>) |
| to use to populate the items.<br> <b>Note:</b> if a combo |
| widget saves its value to an attribute and the present element on |
| the widget descriptor is not set, the attribute will be checked for |
| non-empty <code>choice</code> or <code>items-from</code> values, |
| which then will be used to populate the combo items; the <code>items-from</code> |
| field on the widget is for added flexibility (for instance, linking |
| up a target property with another property providing the list of |
| choices).</td> |
| </tr> |
| <tr> |
| <td><code>fixed-text</code></td> |
| <td>applies to the <code>label</code> and (read-only) <code>text</code> |
| widgets; text is resolved once and remains constant thereafter.</td> |
| </tr> |
| <tr> |
| <td><code>dynamic-text</code></td> |
| <td>applies only to (read-only) <code>text</code> widgets; text |
| is resolved each time there is an update of a value from the Launch |
| Tab.</td> |
| </tr> |
| </table> |
| <br> |
| |
| <p>Widget foreground, background (color) and font are in principle |
| configurable, though the font setting may not always work. The tooltip |
| will appear on the widget whereever it usually does.</p> |
| <br> |
| |
| <h4 align="CENTER"> |
| <a name="ButtonGroup">Button Group</a> |
| </h4> |
| |
| <img alt="ButtonGroup" src="images/20button-group.jpeg" /> |
| |
| <p> |
| The button group provides an explicit selection of a single value |
| based on a mutually exclusive set of button labels. Just as with the |
| <code>widget</code> |
| , one uses <i>saveValueTo</i> to indicate what property or attribute |
| value to set. The |
| <code>button</code> |
| elements, which can be arranged horizontally or vertically using the <i>style</i> |
| attribute (SWT.HORIZONTAL, SWT.VERTICAL), will be |
| <code>widget</code> |
| elements whose type is either <i>checkbox</i> or <i>radiobutton</i> |
| (if a different widget type is given, an error will occur); the <i>saveValueTo</i> |
| attribute on the individual button will be ignored. Radio buttons are |
| "sticky" in the sense that once they are selected, they cannot be |
| "unchecked" until a different button in the group is selected. It is |
| thus advisable always to provide a default value for the property or |
| attribute associated with the group value if radio buttons are used. |
| If check boxes are used, they can be individually checked on and off |
| (one at a time), thus allowing for the value of the group to be |
| potentially empty. |
| </p> |
| |
| <p>Below is an example of a button group using radio buttons, |
| disposed vertically.</p> |
| <br> |
| <img alt="ButtonGroupExample" src="images/ButtonGroup.jpeg" /> |
| <br> |
| <br> |
| |
| <h4 align="CENTER"> |
| <a name="Browse">Browse</a> |
| </h4> |
| |
| <img alt="BrowseType" src="images/190browse.jpeg" /> |
| |
| <p> |
| A special widget-pair -- a text widget followed by a push button -- |
| allowing the user to browse directories or files from either the local |
| or remote file system. Aside from the fields shared with the |
| <code>widget</code> |
| type, we note the following: |
| </p> |
| <br> |
| <table align="center" rules="all"> |
| <tr> |
| <td><i>directory</i></td> |
| <td>limit the <code>browse</code> widget to directories.</td> |
| </tr> |
| <tr> |
| <td><i>uri</i></td> |
| <td>have the <code>browse</code> widget set a URI as the text |
| (default is false, in which case the text is an absolute path).</td> |
| </tr> |
| <tr> |
| <td><i>localOnly</i></td> |
| <td>limit the <code>browse</code> widget to the local file |
| system (otherwise, the choice of local or remote is enabled).</td> |
| </tr> |
| </table> |
| <br> |
| <p> |
| Control ids, |
| <code>layout</code> |
| and |
| <code>control-state</code> |
| are specified as belonging to either the text or the button. Style |
| applies only to the text widget. |
| </p> |
| <br> |
| |
| <h4 align="CENTER"> |
| <a name="Action">Action Button</a> |
| </h4> |
| |
| <br> |
| <img alt="ActionType" src="images/191push-button.jpeg" /> |
| <br> |
| |
| <p> |
| A push button which is associated with a |
| <code>button-action</code> |
| command. The |
| <code>button-action</code> |
| specifies the name of a command ( |
| <code>action</code> |
| ) defined in the XML; this can be one of the |
| <code>start-up</code> |
| or |
| <code>shut-down</code> |
| commands, or a special |
| <code>button-action</code> |
| command (<b>note </b>that it cannot, however, be a <a href="#Note2"><b>(uu)id</b> |
| </a> command, as the Launch Tab is not coupled to any submitted jobs); the |
| <code>button-action-type</code> |
| also indicates whether the variable affected by the command should be |
| cleared prior to the command execution, and whether the launch tab |
| should be refreshed (default is true) afterward. |
| </p> |
| <br> |
| |
| <h4 align="CENTER"> |
| <a name="AttributeViewer">Attribute Viewer</a> |
| </h4> |
| |
| <img alt="AttributeViewerType" src="images/21attribute-viewer.jpeg" /> |
| |
| <p> |
| The last of the widget types, the |
| <code>attribute-viewer</code> |
| , displays visible properties and attributes in a table or tree |
| viewer, with the value columns as editable widgets. The type of cell |
| editor is determined implicitly here by the type of the property or |
| attribute; the presence of a combo editor is determined by an |
| attribute whose |
| <code>choice</code> |
| or |
| <code>items-from</code> |
| element is defined. (This is the only viable possibility for a combo |
| in the table, as there would also need to be a target to save the |
| selected item to, viz., the value of this same attribute.) |
| </p> |
| |
| <p>Each entry in the viewer is provided with a checkbox to indicate |
| which attributes have been chosen (and thus marked valid). A checkbox |
| is also placed above the viewer to allow for filtering the unchecked |
| items. The viewer outputs its values to the environment. The name of |
| the viewer is also stored against a string of all selected values, |
| constructed from a template (further discussion at the end of this |
| section).</p> |
| |
| <p> |
| The viewer <i>type</i> can be either "table" or "tree"; in the case of |
| the tree, the type, default, status and description fields of the |
| attribute or property are displayed as collapsible child rows hanging |
| off the main name entry. Either table or tree can have up to <b>six</b> |
| columns whose names/types are based on the following property and |
| attribute fields: |
| </p> |
| |
| <table align="center" rules="all"> |
| <tr> |
| <td><b>Name</b></td> |
| </tr> |
| <tr> |
| <td><b>Value</b></td> |
| </tr> |
| <tr> |
| <td><b>Type</b></td> |
| </tr> |
| <tr> |
| <td><b>Status</b></td> |
| </tr> |
| <tr> |
| <td><b>Default</b></td> |
| </tr> |
| <tr> |
| <td><b>Description</b></td> |
| </tr> |
| </table> |
| <br> |
| <p> |
| These can appear in any order. There is a slight difference in how |
| these columns are treated, depending upon whether <i>tree</i> or <i>table</i> |
| is chosen as type. For <b>table</b> type, the value of the field |
| simply appears in the corresponding column: |
| </p> |
| <br> |
| <img alt="Table" src="images/Table.jpg" /> |
| <br> |
| <br> |
| <p> |
| In the <b>tree</b> type, the |
| <code>name</code> |
| field displays all fields except |
| <code>value</code> |
| as children; |
| <code>value</code> |
| always appears as a separate column on the same row as the name (it |
| contains the cell editor). If the viewer also has the column |
| corresponding to the child field, that field value appears in that |
| column; |
| </p> |
| |
| <br> |
| <img alt="Tree1" src="images/Tree1.jpg" /> |
| <br> |
| <br> |
| |
| <p> |
| Otherwise, the child fields display their value in the value column |
| along with the actual |
| <code>value</code> |
| field: |
| </p> |
| |
| <br> |
| <img alt="Tree2" src="images/Tree2.jpg" /> |
| <br> |
| <br> |
| |
| <p> |
| Tooltips for the column appear when hovering over the column header. (<i>A |
| caveat on the foreground and background settings for this type: they |
| are exposed here but may not work as expected: these are currently |
| applied to the column through the cell label provider; foreground |
| does not change the font color but applies the color to the entire |
| cell.</i>) |
| </p> |
| |
| <p> |
| <i>sort</i> is done on the name field/column. This is a toggle sort, |
| such that alternate clicks reverse the sort direction. <i>headerVisible</i> |
| and <i>linesVisible</i> control the look of the table. <i>tooltipEnabled</i> |
| enables column-viewer label provider tooltip support. The tooltip |
| defined for an attribute will appear on the row of the table or the |
| parent node of the tree. The attribute description will also appear as |
| a tooltip on the description columns if they exist. |
| </p> |
| |
| <p> |
| Control over the actual contents of the viewer is achieved through the |
| <code>viewer-items-type</code> |
| ; one can here explicitly list properties or attributes for inclusion |
| using the |
| <code>include</code> |
| element, or one can set one or both of the <i>all</i> attributes; in |
| the latter case, attributes and properties can be excluded from the |
| set using |
| <code>exclude</code> |
| . The <i>initialCheckAll</i> field, which defaults to true, indicates |
| that if the attribute or property is new or undefined from a previous |
| session, it will appear as checked; turn this off if you want new |
| attributes or properties to appear at first as unchecked. |
| </p> |
| |
| <p> |
| A property bearing the name given to this widget is written to the |
| environment at each update/refresh; the value of the property is a |
| string formed from the selected (checked) rows of the table using the |
| <code>template-type</code> |
| (the |
| <code>value</code> |
| element). The <i>pattern</i> can be any literal string embedding <i>@name</i> |
| and <i>@value</i> tags as placeholders for the row's name and value; |
| for instance, '@name="@value"' would generate a string of name-value |
| assignments. The default <i>separator</i> between patterned segments |
| is a single space. |
| </p> |
| |
| <p>Below is a fuller example of the tree version of the attribute |
| viewer. The text area above it displays the current value of its |
| templated pattern string. The child rows reporting status, default, |
| etc., cannot be selected or unselected (only the parent row can). |
| Hovering over the name (here walltime) displays the tooltip.</p> |
| <br> |
| <img alt="AttributeViewerTree" src="images/AttribViewerTree.jpeg" /> |
| <br> |
| |
| <h3> |
| <a name="MonitorType">The Monitor Data Type</a> |
| </h3> |
| |
| <p>At present, the monitor component has less elements to define; |
| this may change in future releases, when certain hard-coded features |
| of the driver may be handed over to the client for configuration.</p> |
| |
| <img alt="MonitorType" src="images/24monitor.jpeg" /> |
| |
| <p> |
| This component can be furnished with its own set of properties |
| (currently unused). What is necessary to set at present is only the <i>schedulerType</i> |
| attribute (e.g., "PBS"). <i>refreshFrequencyInSeconds</i> defaults to |
| 60, but this can be changed according to the needs of the user. Be |
| aware that too low a setting will probably not work, as the command on |
| an average sized system will take upwards of five seconds to complete |
| (XML is being streamed to the driver and a |
| <code>diff</code> |
| is sent back to the client ). |
| </p> |
| |
| <p> |
| If the |
| <code>driver</code> |
| element is configured, then the default behavior, which is to stage |
| over the necessary driver scripts to the <i>.eclipsesettings</i> |
| directory in the user's home, is overridden by connecting to a |
| pre-existent installation. This can be specified either using the |
| <code>url</code> |
| , or a combination of the |
| <code>name</code> |
| , |
| <code>path</code> |
| and |
| <code>args</code> |
| elements. |
| </p> |
| <hr> |
| <a href="toc.html">Back to Table of Contents</a> |
| </body> |
| </html> |