bundles/org.eclipse.test.performance.ui/readme.html - gerrit/platform/eclipse.platform.releng.buildtools - Git at Google

 <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html lang="en">
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
 <title>org.eclipse.test.performance.ui readme</title>
 </head>

 <p>
 	<b><font size="+2">README</font></b><br>
 </p>
 <p>
 	Last updated: October 26, 2005<br>
 </p>
 <p>
 	<b>Introduction</b><br> <br> This readme describes how to
 	create performance charts and tables similar to those available on the
 	eclipse platform download pages using the stand-alone java program <b>org.eclipse.test.performance.ui.Main</b>
 	stored in this project. Although the program is currently geared to
 	generating results for the Eclipse Platform builds, the program may
 	work for you given the following conditions:
 </p>
 <ul>
 	<li>The &quot;eclipse.perf.config&quot; system property used to
 		run tests used three keys config, build and jvm
 		<ul>
 			<li>for example,
 				"eclipse.perf.config=config=relengbuildwin2;build=I20050219-1500;jvm=sun1.4.2_06".
 				<br> <br>
 			</li>
 		</ul>
 	</li>
 	<li>Values for the "build" key begin with either an 'I' or 'N'
 		with the exception of the reference builds (for example, 3.0).
 		<ul>
 			<li>The line graphs plot the reference build, all builds
 				starting with 'I' and the seven 'N' builds preceding the current
 				build.</li>
 			<li>This can be overriden with the use of the
 				&quot;-current.prefix&quot; parameter where you can specify a
 				comma-separated list of prefixes for build ids to include in the
 				main line graph.<br> <br>
 			</li>
 		</ul>
 	</li>
 	<li>Scenario names all contain a prefix
 		&quot;&lt;component&gt;.test&quot;
 		<ul>
 			<li>For example, it is assumed that the scenario
 				&quot;org.eclipse.help.tests.performance.appserver.TestAppserver#testAppserverCycle()&quot;
 				belongs to the &quot;org.eclipse.help&quot; component.</li>
 			<li>Unique scenario prefixes are derived from all the scenarios
 				in the database and are used to group results.</li>
 		</ul>
 	</li>
 </ul>
 <p>
 	<br> For purposes of this document, the term &quot;<a
 		href="images/descriptions.html#fp">fingerprint</a>&quot; will be used
 	to refer to a bar graph which represents the performance summary at a
 	global or component level for each test configuration.<br> <br>
 	The term &quot;<a href="images/descriptions.html#sst">scenario
 		status table</a>&quot; will be used to refer to a table of scenarios with
 	green checks and red x indicators below a component fingerprint. This
 	table gives an overall view of the status for all scenarios for a
 	component for all configurations.<br>
 </p>
 <p>
 	The term &quot;<a href="images/descriptions.html#sr">scenario
 		results</a>&quot; will be used to refer to a web page comparing current
 	performance measurements to the baseline measurements for one scenario
 	on one configuration. The web page displays the raw data for the
 	current and baseline measurements and also displays line graphs showing
 	the measured values over the course of builds between the current and
 	baseline.
 </p>
 <p>
 	<br> <b>Requirements</b>
 </p>
 <ul>
 	<li>Windows or Linux (x86) operating system</li>
 	<li>org.eclipse.releng.basebuilder</li>
 	<li>JVM 11 installed and on path</li>
 </ul>
 <p>
 	<b>Running the program</b>
 </p>
 <p>
 	From a terminal or dos prompt,<br>
 </p>
 <OL>
 	<LI>cd to
 		org.eclipse.releng.basebuilder\plugins\org.eclipse.test.performance.ui\scripts<BR>
 	</LI>
 	<LI>If running on Linux, execute &quot;chmod 755 genresults&quot;</LI>
 	<LI>Execute the following command:<b><br> <br>
 			genresults -baseline &lt;baseline build id&gt; -current &lt;current
 			build id&gt; -config &lt;comma-separated list of configs&gt; -jvm
 			&lt;jvm name&gt; -output &lt;output dir&gt; [-config.properties
 			&lt;properties&gt;] [highlight &lt;buildid patterns&gt;]
 			[scenario.pattern &lt;scenario prefix patterns&gt;]
 			[-fingerprints][-scenarioresults][-current.prefix &lt;comma-separated
 			list of build id prefixes&gt;][-baseline.prefix &lt;comma-separated
 			list of build id prefixes&gt;]</b></LI>
 </OL>
 <p>
 	<br> Parameter descriptions:
 </p>
 <blockquote>
 	<ul>
 		<li>-baseline - baseline build id. A previous value used in the
 			&quot;build&quot; key in the eclipse.perf.config system property or
 			the eclipse.perf.assertAgainst property.</li>
 		<li>-current - current build id. The value used in the
 			&quot;build&quot; key in the eclipse.perf.config system property.</li>
 		<li>-config - a comma-separated list of names of configurations
 			for which to produce results. The values used should be the same as
 			values specified for the &quot;config&quot; key in the
 			eclipse.perf.config system property.</li>
 		<li>-jvm -jvm
 			description. The value used in the &quot;jvm&quot; key in the
 			eclipse.perf.config system property.
 		</li>
 		<li>-output - path to an output directory for fingerprints and
 			scenarios for each configuration specified in -config parameter.
 			Fingerprint gifs and html files are generated in the directory
 			specified, configuration specific subdirectories are created to store
 			scenario results and line graphs.</li>
 	</ul>
 </blockquote>
 <p>
 	<br> Optional arguments:
 </p>
 <blockquote>
 	<ul>
 		<li>-config.properties - semi-colon separated list of: config,
 			alternate config description. The value should be quoted if there are
 			spaces in the value.<BR>of builds which match any prefix in this
 			list. When omitted, a magenta line is draw on the graph which
 			represents the baseline value specified in the -baseline parameter.
 		</li>
 		<li>-highlight - Comma-separated list of build Id
 			prefixes(excluding 'I' and 'N'), which are used to add and highlight
 			the most recent matching buildId on line graphs.</li>
 		<li>-scenario.pattern - Scenario prefix pattern used to query
 			performance results database. Can be used to generate results for a
 			single component or subset of scenarios.</li>
 		<li>-fingerprints or -scenarioresults - use one or the other to
 			generate fingerprints or scenario results only. Not specifying either
 			will execute both.</li>
 		<li>-baseline.prefix - semi-colon separated list of: build id
 			prefixes used in eclipse.perf.assertAgainst property or
 			eclipse.perf.config system property. Values used to generate a second
 			line graph representing changes in repeated baseline test run.</li>
 		<li>-current.prefix - semi-colon separated list of: build id
 			prefixes used in eclipse.perf.config system property. Values here
 			used to override default of &quot;N,I&quot;. Used to select build
 			id's to include in main line graph.</li>
 	</ul>
 </blockquote>
 </html>
	<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
	<html lang="en">
	<head>
	<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
	<title>org.eclipse.test.performance.ui readme</title>
	</head>

	<p>
	<b><font size="+2">README</font></b><br>
	</p>
	<p>
	Last updated: October 26, 2005<br>
	</p>
	<p>
	<b>Introduction</b><br> <br> This readme describes how to
	create performance charts and tables similar to those available on the
	eclipse platform download pages using the stand-alone java program <b>org.eclipse.test.performance.ui.Main</b>
	stored in this project. Although the program is currently geared to
	generating results for the Eclipse Platform builds, the program may
	work for you given the following conditions:
	</p>
	<ul>
	<li>The "eclipse.perf.config" system property used to
	run tests used three keys config, build and jvm
	<ul>
	<li>for example,
	"eclipse.perf.config=config=relengbuildwin2;build=I20050219-1500;jvm=sun1.4.2_06".
	<br> <br>
	</li>
	</ul>
	</li>
	<li>Values for the "build" key begin with either an 'I' or 'N'
	with the exception of the reference builds (for example, 3.0).
	<ul>
	<li>The line graphs plot the reference build, all builds
	starting with 'I' and the seven 'N' builds preceding the current
	build.</li>
	<li>This can be overriden with the use of the
	"-current.prefix" parameter where you can specify a
	comma-separated list of prefixes for build ids to include in the
	main line graph.<br> <br>
	</li>
	</ul>
	</li>
	<li>Scenario names all contain a prefix
	"<component>.test"
	<ul>
	<li>For example, it is assumed that the scenario
	"org.eclipse.help.tests.performance.appserver.TestAppserver#testAppserverCycle()"
	belongs to the "org.eclipse.help" component.</li>
	<li>Unique scenario prefixes are derived from all the scenarios
	in the database and are used to group results.</li>
	</ul>
	</li>
	</ul>
	<p>
	<br> For purposes of this document, the term "<a
	href="images/descriptions.html#fp">fingerprint</a>" will be used
	to refer to a bar graph which represents the performance summary at a
	global or component level for each test configuration.<br> <br>
	The term "<a href="images/descriptions.html#sst">scenario
	status table</a>" will be used to refer to a table of scenarios with
	green checks and red x indicators below a component fingerprint. This
	table gives an overall view of the status for all scenarios for a
	component for all configurations.<br>
	</p>
	<p>
	The term "<a href="images/descriptions.html#sr">scenario
	results</a>" will be used to refer to a web page comparing current
	performance measurements to the baseline measurements for one scenario
	on one configuration. The web page displays the raw data for the
	current and baseline measurements and also displays line graphs showing
	the measured values over the course of builds between the current and
	baseline.
	</p>
	<p>
	<br> <b>Requirements</b>
	</p>
	<ul>
	<li>Windows or Linux (x86) operating system</li>
	<li>org.eclipse.releng.basebuilder</li>
	<li>JVM 11 installed and on path</li>
	</ul>
	<p>
	<b>Running the program</b>
	</p>
	<p>
	From a terminal or dos prompt,<br>
	</p>
	<OL>
	<LI>cd to
	org.eclipse.releng.basebuilder\plugins\org.eclipse.test.performance.ui\scripts<BR>
	</LI>
	<LI>If running on Linux, execute "chmod 755 genresults"</LI>
	<LI>Execute the following command:<b><br> <br>
	genresults -baseline <baseline build id> -current <current
	build id> -config <comma-separated list of configs> -jvm
	<jvm name> -output <output dir> [-config.properties
	<properties>] [highlight <buildid patterns>]
	[scenario.pattern <scenario prefix patterns>]
	[-fingerprints][-scenarioresults][-current.prefix <comma-separated
	list of build id prefixes>][-baseline.prefix <comma-separated
	list of build id prefixes>]</b></LI>
	</OL>
	<p>
	<br> Parameter descriptions:
	</p>
	<blockquote>
	<ul>
	<li>-baseline - baseline build id. A previous value used in the
	"build" key in the eclipse.perf.config system property or
	the eclipse.perf.assertAgainst property.</li>
	<li>-current - current build id. The value used in the
	"build" key in the eclipse.perf.config system property.</li>
	<li>-config - a comma-separated list of names of configurations
	for which to produce results. The values used should be the same as
	values specified for the "config" key in the
	eclipse.perf.config system property.</li>
	<li>-jvm -jvm
	description. The value used in the "jvm" key in the
	eclipse.perf.config system property.
	</li>
	<li>-output - path to an output directory for fingerprints and
	scenarios for each configuration specified in -config parameter.
	Fingerprint gifs and html files are generated in the directory
	specified, configuration specific subdirectories are created to store
	scenario results and line graphs.</li>
	</ul>
	</blockquote>
	<p>
	<br> Optional arguments:
	</p>
	<blockquote>
	<ul>
	<li>-config.properties - semi-colon separated list of: config,
	alternate config description. The value should be quoted if there are
	spaces in the value.<BR>of builds which match any prefix in this
	list. When omitted, a magenta line is draw on the graph which
	represents the baseline value specified in the -baseline parameter.
	</li>
	<li>-highlight - Comma-separated list of build Id
	prefixes(excluding 'I' and 'N'), which are used to add and highlight
	the most recent matching buildId on line graphs.</li>
	<li>-scenario.pattern - Scenario prefix pattern used to query
	performance results database. Can be used to generate results for a
	single component or subset of scenarios.</li>
	<li>-fingerprints or -scenarioresults - use one or the other to
	generate fingerprints or scenario results only. Not specifying either
	will execute both.</li>
	<li>-baseline.prefix - semi-colon separated list of: build id
	prefixes used in eclipse.perf.assertAgainst property or
	eclipse.perf.config system property. Values used to generate a second
	line graph representing changes in repeated baseline test run.</li>
	<li>-current.prefix - semi-colon separated list of: build id
	prefixes used in eclipse.perf.config system property. Values here
	used to override default of "N,I". Used to select build
	id's to include in main line graph.</li>
	</ul>
	</blockquote>
	</html>