rms/org.eclipse.ptp.rm.jaxb.doc.isv/html/FilesAndScripts.html - ptp/org.eclipse.ptp - Git at Google

 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
 <title>Files and Scripts</title>
 <link rel="stylesheet" type="text/css" href="help.css">
 </head>
 <body>
 	<h2>
 		Files and Scripts
 	</h2>

 	This section is used to define the files and scripts that are necessary for submitting a job
 	to the target system.

 	<h3>
 		<a name="Files">Managed Files</a>
 	</h3>

 	<p>
 		A "managed file" is a file that may be required for the job, but
 		which may not be present on the host on which the job will run.
 		These files may either be external to Eclipse or may be generated from
 		the environment in conjunction with the job
 		submission, but in either case need to be copied to the target system just prior
 		to it.
 	</p>

 	<img alt="ManagedFiles" src="images/05managed-files.jpeg" />

 	<p>
 		Managed files are added to the definition in groups determined by
 		their shared staging location, specified using the <code>file-staging-location</code> element,
 		which is a path relative to the working
 		directory of the connection. Managed file properties are as follows:
 	</p>

 	<table cellpadding="5" border="1" rules="all">
 		<tr>
 			<th>Property</th>
 			<th>Description</th>
 			<th>Default</th>
 		</tr>
 		<tr>
 			<td><i>name</i></td>
 			<td>
 				A mandatory name for the managed file.
 			</td>
 			<td>N/A</td>
 		</tr>
 		<tr>
 			<td><i>uniqueIdPrefix</i></td>
 			<td>
 				Specify that multiple copies of a generated file should be distinguishable.
 			</td>
 			<td>false</td>
 		</tr>
 		<tr>
 			<td><i>resolveContents</i></td>
 			<td>
 				Pass the <code>contents</code> element through the attribute resolver (see below).
 			</td>
 			<td>true</td>
 		</tr>
 		<tr>
 			<td><i>deleteSourceAfterUse</i></td>
 			<td>
 				Specify that local copies of generated files should be retained.
 			</td>
 			<td>false</td>
 		</tr>
 		<tr>
 			<td><i>deleteTargetAfterUse</i></td>
 			<td>
 				Specify that the file should be deleted from the target system after use (not currently
 				implemented in the general case).
 			</td>
 			<td>false</td>
 		</tr>
 	</table>

 	<p>
 		<b>Note:</b> The batch script is staged by default to
 		<i>.eclipsesettings</i> in the user's home directory and deleted after the submit call returns.
 		However,
 		it is possible to use the
 		<code>file-staging-location</code>
 		on the
 		<code>script</code>
 		element to set the batch script path explicitly.
 	</p>

 	<p>
 		A managed file definition allows one of three possibilities as
 		to specifying content:
 	</p>

 	<table cellpadding="5" border="1" rules="all">
 		<tr>
 			<th>Element</th>
 			<th>Description</th>
 		</tr>
 		<tr>
 			<td><code>path</code></td>
 			<td>
 				Specifies the location of an external file, which can be either a hard-coded path or a reference to an
 				attribute value.
 			</td>
 		</tr>
 		<tr>
 			<td><code>contents</code></td>
 			<td>
 				Specifies the file contents as a string.
 				In this case the <i>resolveContents</i> property is used to indicate whether to pass this string through the
 				attribute resolver, substituting any references to
 				attribute values it may contain. However, this string should not contain "${...}" sequences which
 				do not actually refer to Eclipse variables (such as batch script
 				variables), or the resolution will fail. If <i>resolveContents</i> is
 				set to false, you can still provide this text element with a single
 				reference to an attribute, in which case the value of that
 				attribute will be used <i>as is</i>, without further
 				dereferencing.
 			</td>
 		</tr>
 		<tr>
 			<td><code>line</code></td>
 			<td>
 				Specifies the file contents using <code>line-type</code> elements.
 				This is the preferred and most flexible way to represent the
 				contents of a file to be generated. The <code>line-type</code> type is discussed in more detail
 				<a href="#LineArg">below</a>.
 			</td>
 		</tr>
 	</table>

 	<p>When the submit call is executed, managed files are generated if necessary,
 		then their target paths are determined as follows:</p>

 	<ul>
 		<li>If the <code>path</code> element is used, the target path
 			becomes <i>staging_directory / name_of_pre-existing_file</i>.</li>
 		<li>If the <code>content</code> or <code>line</code> elements are
 			used, the target path becomes <i>staging_directory/
 				[uniqueIdPrefix]managed-file-name</i>).</li>
 		<li>An attribute is placed in the environment whose <i>name</i> is
 			the managed-file <i>name</i> and whose <i>value</i> is this target
 			path.</li>
 	</ul>

 	<h3>
 		<a name="Script">Scripts</a>
 	</h3>

 	<p>
 		Schedulers such as PBS
 		or LoadLeveler normally use a job script in order to specify the resources
 		required to launch a job (although they can be configured to work without a batch script.)
 		Using a script file generally provides the most flexibility in configuring the job submission.
 		A
 		<code>script-type</code>
 		file is just a special
 		case of the
 		<code>managed-file-type</code>
 		type
 	</p>

 	<img alt="ScriptType" src="images/06script.png" />

 	<p>
 		If the script is specified in the configuration, its path is
 		automatically added to the list of managed files to be staged to the
 		appropriate directory (by default <i>.eclipsesettings</i>, or
 		as indicated by the
 		<code>file-staging-location</code>
 		element), so there is no need to include a script entry explicitly
 		under the
 		<code>managed-files</code>
 		element. The following properties are available for scripts:
 	</p>

 	<table cellpadding="5" border="1" rules="all">
 		<tr>
 			<th>Property</th>
 			<th>Description</th>
 			<th>Default</th>
 		</tr>
 		<tr>
 			<td><i>insertEnvironmentAfter</i></td>
 			<td>
 				Supplies a line number indicating where in the
 				script to add any extra environment variables set through the <b>Environment</b>
 				tab provided in the launch configuration. This provides
 				control over whether these should overwrite previously
 				defined environment variable values.
 			</td>
 			<td>-1 (don't insert)</td>
 		</tr>
 		<tr>
 			<td><i>deleteAfterSubmit</i></td>
 			<td>
 				Indicates that the script target should not
 				be retained. Unlike the
 				<code>managed-file-type</code>, however, the local copy of the generated script is always deleted.
 			</td>
 			<td>true</td>
 		</tr>
 	</table>

  	<p>
  		A reserved attribute,
 				<code>managed_file_for_script</code>, is used to reference the script's path on the target
 				resource in the submit command, e.g.:
 	</p>
 	<p>
 		<code>
 			&lt;arg&gt;qsub&lt;/arg&gt;<br>
 			&lt;arg&gt;${ptp_rm:managed_file_for_script#value}&lt;/arg&gt;
 		</code>
 	</p>

 	<p>
 		<b>Note</b>: If the
 		<code>import</code>
 		tab (see <a href="#LaunchTab">below</a>) is used to provide an
 		external or workspace edited batch script to the run, nothing extra
 		need be done in the configuration. The
 		path for the script is handled automatically.
 	</p>

 	<h3>
 		<a name="LineArg">Lines</a>
 	</h3>

 	<p>
 		The <code>line-type</code> is used to explictly specify lines for a script or managed file. Each
 		<code>line-type</code>
 		element contains zero or more
 		<code>arg-type</code>
 		elements. The text specified by the <code>arg-type</code> elements is placed on a single line, separated by
 		whitespace. The sequence is terminated by a line separator.
 	</p>

 	<img alt="Line" src="images/07line-arg.png" />

 	<h3>
 		<a name="Arguments">Arguments</a>
 	</h3>

 	<p>
 		An
 		<code>arg-type</code>
 		element is used for script and managed file content as well as in the
 		definition of commands. Its text element contains the actual argument
 		string which is passed to the attribute resolver before being written
 		out. The default behavior of the argument resolver is not to
 		include arguments whose values are <code>null</code>.
 		The <code>arg-type</code> type provides the following properties:
 	</p>

 	<table cellpadding="5" border="1" rules="all">
 		<tr>
 			<th>Property</th>
 			<th>Description</th>
 			<th>Default</th>
 		</tr>
 		<tr>
 			<td><i>attribute</i></td>
 			<td>
 				<p>
 				Apply the match to the value of the referenced attribute
 				rather than the content of the element. For example,
 				a boolean attribute could be used to control the inclusion/exclusion of the argument as
 				follows:</p>
 				</p>
 				<code> &lt;arg attribute="useFlags" isUndefinedIfMatches="false"&gt;${ptp_rm:flag#value}&lt;/arg&gt; </code>
 			</td>
 			<td>""</td>
 		</tr>
 		<tr>
 			<td><i>isUndefinedIfMatches</i></td>
 			<td>
 				<p>
 				 Specifies a regular
 				expression that is compared to the argument after resolution. If the regular
 				expression matches, then the argument will be omitted from the result. For instance, if a
 				flag should not appear when the value it precedes is an empty string,
 				one could write:
 				</p>
 				<code> &lt;arg isUndefinedIfMatches="-f"&gt;-f
 					${ptp_rm:flag#value}&lt;/arg&gt; </code>

 				<p>For the purposes of matching, trailing whitespace is trimmed
 					from the resolved argument, so there is no need to specify this as
 					part of the regex used to match.</p>
 			</td>
 			<td>""</td>
 		</tr>
 		<tr>
 			<td><i>resolve</i></td>
 			<td>
 				Indicates that the text element should be treated as a literal and no
 				attribute resolution performed on the string.. The <i>resolve</i> property must be set
 				to false if the string contains batch-type variables (e.g., ${HOME}) which should be
 				resolved by the remote shell and not inside the Eclipse client.
 			</td>
 			<td>true</td>
 		</tr>
 	</table>

 	<h3>
 		<a name="JustInTimeEval">"Just-in-time" resolution of <i>@jobId</i>
 			and <code>managed-file</code> paths</a>
 	</h3>

 	<p>
 		<i>@jobId</i> is a special attribute name designating the runtime id
 		for a job instance. In the lifecycle of the run/launch (submit) call,
 		this value begins as an internally generated unique id (uuid) which then is
 		replaced by the real job id <i>after</i> the job has been submitted.
 	</p>

 	<p>
 		The <i>@jobId</i> attribute, along with the target paths for
 		<code>managed-file</code>
 		elements, are not known at configuration time (i.e., before the user
 		hits "Run"). While the former is made visible to the parsers and the
 		returned status object of the submit command, neither is in the scope
 		of (available for reference in) other managed files or the
 		<code>script</code>
 		element, because these latter elements are generated just prior to the
 		actual submission.
 	</p>

 	<p>
 		If the
 		<code>script</code>
 		needs to refer to the <i>@jobId</i>, it must do so via an environment variable
 		made available by the particular scheduler it is written for. An
 		example of how to reference the target path of a
 		<code>managed-file</code>
 		inside the
 		<code>script</code>
 		is included in the <a href="http://download.eclipse.org/tools/ptp/docs/JAXBDemo.pdf">tutorial slides</a>. This
 		essentially involves defining an environment variable in the submission command's
 		environment, with a reference to the
 		<code>managed-file</code>
 		path attribute as its value, and then using this environment variable
 		inside the
 		<code>script</code>
 		.
 	</p>

 </body>
 </html>
	<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
	<html>
	<head>
	<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
	<title>Files and Scripts</title>
	<link rel="stylesheet" type="text/css" href="help.css">
	</head>
	<body>
	<h2>
	Files and Scripts
	</h2>

	This section is used to define the files and scripts that are necessary for submitting a job
	to the target system.

	<h3>
	<a name="Files">Managed Files</a>
	</h3>

	<p>
	A "managed file" is a file that may be required for the job, but
	which may not be present on the host on which the job will run.
	These files may either be external to Eclipse or may be generated from
	the environment in conjunction with the job
	submission, but in either case need to be copied to the target system just prior
	to it.
	</p>

	<img alt="ManagedFiles" src="images/05managed-files.jpeg" />

	<p>
	Managed files are added to the definition in groups determined by
	their shared staging location, specified using the <code>file-staging-location</code> element,
	which is a path relative to the working
	directory of the connection. Managed file properties are as follows:
	</p>

	<table cellpadding="5" border="1" rules="all">
	<tr>
	<th>Property</th>
	<th>Description</th>
	<th>Default</th>
	</tr>
	<tr>
	<td><i>name</i></td>
	<td>
	A mandatory name for the managed file.
	</td>
	<td>N/A</td>
	</tr>
	<tr>
	<td><i>uniqueIdPrefix</i></td>
	<td>
	Specify that multiple copies of a generated file should be distinguishable.
	</td>
	<td>false</td>
	</tr>
	<tr>
	<td><i>resolveContents</i></td>
	<td>
	Pass the <code>contents</code> element through the attribute resolver (see below).
	</td>
	<td>true</td>
	</tr>
	<tr>
	<td><i>deleteSourceAfterUse</i></td>
	<td>
	Specify that local copies of generated files should be retained.
	</td>
	<td>false</td>
	</tr>
	<tr>
	<td><i>deleteTargetAfterUse</i></td>
	<td>
	Specify that the file should be deleted from the target system after use (not currently
	implemented in the general case).
	</td>
	<td>false</td>
	</tr>
	</table>

	<p>
	<b>Note:</b> The batch script is staged by default to
	<i>.eclipsesettings</i> in the user's home directory and deleted after the submit call returns.
	However,
	it is possible to use the
	<code>file-staging-location</code>
	on the
	<code>script</code>
	element to set the batch script path explicitly.
	</p>

	<p>
	A managed file definition allows one of three possibilities as
	to specifying content:
	</p>

	<table cellpadding="5" border="1" rules="all">
	<tr>
	<th>Element</th>
	<th>Description</th>
	</tr>
	<tr>
	<td><code>path</code></td>
	<td>
	Specifies the location of an external file, which can be either a hard-coded path or a reference to an
	attribute value.
	</td>
	</tr>
	<tr>
	<td><code>contents</code></td>
	<td>
	Specifies the file contents as a string.
	In this case the <i>resolveContents</i> property is used to indicate whether to pass this string through the
	attribute resolver, substituting any references to
	attribute values it may contain. However, this string should not contain "${...}" sequences which
	do not actually refer to Eclipse variables (such as batch script
	variables), or the resolution will fail. If <i>resolveContents</i> is
	set to false, you can still provide this text element with a single
	reference to an attribute, in which case the value of that
	attribute will be used <i>as is</i>, without further
	dereferencing.
	</td>
	</tr>
	<tr>
	<td><code>line</code></td>
	<td>
	Specifies the file contents using <code>line-type</code> elements.
	This is the preferred and most flexible way to represent the
	contents of a file to be generated. The <code>line-type</code> type is discussed in more detail
	<a href="#LineArg">below</a>.
	</td>
	</tr>
	</table>

	<p>When the submit call is executed, managed files are generated if necessary,
	then their target paths are determined as follows:</p>

	<ul>
	<li>If the <code>path</code> element is used, the target path
	becomes <i>staging_directory / name_of_pre-existing_file</i>.</li>
	<li>If the <code>content</code> or <code>line</code> elements are
	used, the target path becomes <i>staging_directory/
	[uniqueIdPrefix]managed-file-name</i>).</li>
	<li>An attribute is placed in the environment whose <i>name</i> is
	the managed-file <i>name</i> and whose <i>value</i> is this target
	path.</li>
	</ul>

	<h3>
	<a name="Script">Scripts</a>
	</h3>

	<p>
	Schedulers such as PBS
	or LoadLeveler normally use a job script in order to specify the resources
	required to launch a job (although they can be configured to work without a batch script.)
	Using a script file generally provides the most flexibility in configuring the job submission.
	A
	<code>script-type</code>
	file is just a special
	case of the
	<code>managed-file-type</code>
	type
	</p>

	<img alt="ScriptType" src="images/06script.png" />

	<p>
	If the script is specified in the configuration, its path is
	automatically added to the list of managed files to be staged to the
	appropriate directory (by default <i>.eclipsesettings</i>, or
	as indicated by the
	<code>file-staging-location</code>
	element), so there is no need to include a script entry explicitly
	under the
	<code>managed-files</code>
	element. The following properties are available for scripts:
	</p>

	<table cellpadding="5" border="1" rules="all">
	<tr>
	<th>Property</th>
	<th>Description</th>
	<th>Default</th>
	</tr>
	<tr>
	<td><i>insertEnvironmentAfter</i></td>
	<td>
	Supplies a line number indicating where in the
	script to add any extra environment variables set through the <b>Environment</b>
	tab provided in the launch configuration. This provides
	control over whether these should overwrite previously
	defined environment variable values.
	</td>
	<td>-1 (don't insert)</td>
	</tr>
	<tr>
	<td><i>deleteAfterSubmit</i></td>
	<td>
	Indicates that the script target should not
	be retained. Unlike the
	<code>managed-file-type</code>, however, the local copy of the generated script is always deleted.
	</td>
	<td>true</td>
	</tr>
	</table>

	<p>
	A reserved attribute,
	<code>managed_file_for_script</code>, is used to reference the script's path on the target
	resource in the submit command, e.g.:
	</p>
	<p>
	<code>
	<arg>qsub</arg><br>
	<arg>${ptp_rm:managed_file_for_script#value}</arg>
	</code>
	</p>

	<p>
	<b>Note</b>: If the
	<code>import</code>
	tab (see <a href="#LaunchTab">below</a>) is used to provide an
	external or workspace edited batch script to the run, nothing extra
	need be done in the configuration. The
	path for the script is handled automatically.
	</p>

	<h3>
	<a name="LineArg">Lines</a>
	</h3>

	<p>
	The <code>line-type</code> is used to explictly specify lines for a script or managed file. Each
	<code>line-type</code>
	element contains zero or more
	<code>arg-type</code>
	elements. The text specified by the <code>arg-type</code> elements is placed on a single line, separated by
	whitespace. The sequence is terminated by a line separator.
	</p>

	<img alt="Line" src="images/07line-arg.png" />

	<h3>
	<a name="Arguments">Arguments</a>
	</h3>

	<p>
	An
	<code>arg-type</code>
	element is used for script and managed file content as well as in the
	definition of commands. Its text element contains the actual argument
	string which is passed to the attribute resolver before being written
	out. The default behavior of the argument resolver is not to
	include arguments whose values are <code>null</code>.
	The <code>arg-type</code> type provides the following properties:
	</p>

	<table cellpadding="5" border="1" rules="all">
	<tr>
	<th>Property</th>
	<th>Description</th>
	<th>Default</th>
	</tr>
	<tr>
	<td><i>attribute</i></td>
	<td>
	<p>
	Apply the match to the value of the referenced attribute
	rather than the content of the element. For example,
	a boolean attribute could be used to control the inclusion/exclusion of the argument as
	follows:</p>
	</p>
	<code> <arg attribute="useFlags" isUndefinedIfMatches="false">${ptp_rm:flag#value}</arg> </code>
	</td>
	<td>""</td>
	</tr>
	<tr>
	<td><i>isUndefinedIfMatches</i></td>
	<td>
	<p>
	Specifies a regular
	expression that is compared to the argument after resolution. If the regular
	expression matches, then the argument will be omitted from the result. For instance, if a
	flag should not appear when the value it precedes is an empty string,
	one could write:
	</p>
	<code> <arg isUndefinedIfMatches="-f">-f
	${ptp_rm:flag#value}</arg> </code>

	<p>For the purposes of matching, trailing whitespace is trimmed
	from the resolved argument, so there is no need to specify this as
	part of the regex used to match.</p>
	</td>
	<td>""</td>
	</tr>
	<tr>
	<td><i>resolve</i></td>
	<td>
	Indicates that the text element should be treated as a literal and no
	attribute resolution performed on the string.. The <i>resolve</i> property must be set
	to false if the string contains batch-type variables (e.g., ${HOME}) which should be
	resolved by the remote shell and not inside the Eclipse client.
	</td>
	<td>true</td>
	</tr>
	</table>

	<h3>
	<a name="JustInTimeEval">"Just-in-time" resolution of <i>@jobId</i>
	and <code>managed-file</code> paths</a>
	</h3>

	<p>
	<i>@jobId</i> is a special attribute name designating the runtime id
	for a job instance. In the lifecycle of the run/launch (submit) call,
	this value begins as an internally generated unique id (uuid) which then is
	replaced by the real job id <i>after</i> the job has been submitted.
	</p>

	<p>
	The <i>@jobId</i> attribute, along with the target paths for
	<code>managed-file</code>
	elements, are not known at configuration time (i.e., before the user
	hits "Run"). While the former is made visible to the parsers and the
	returned status object of the submit command, neither is in the scope
	of (available for reference in) other managed files or the
	<code>script</code>
	element, because these latter elements are generated just prior to the
	actual submission.
	</p>

	<p>
	If the
	<code>script</code>
	needs to refer to the <i>@jobId</i>, it must do so via an environment variable
	made available by the particular scheduler it is written for. An
	example of how to reference the target path of a
	<code>managed-file</code>
	inside the
	<code>script</code>
	is included in the <a href="http://download.eclipse.org/tools/ptp/docs/JAXBDemo.pdf">tutorial slides</a>. This
	essentially involves defining an environment variable in the submission command's
	environment, with a reference to the
	<code>managed-file</code>
	path attribute as its value, and then using this environment variable
	inside the
	<code>script</code>
	.
	</p>

	</body>
	</html>