| <!doctype html public "-//w3c//dtd html 4.0 transitional//en"> |
| <html> |
| <head> |
| <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> |
| <title>org.eclipse.test.performance.ui readme</title> |
| </head> |
| |
| <p><b><font size="+2">README</font></b><br> </p><p>last updated: October 26, 2005<br> |
| </p><p><b>Introduction</b><br> <br> This readme describes how to create performance |
| charts and tables similar to those available on the eclipse platform download |
| pages using the stand-alone java program <b>org.eclipse.test.performance.ui.Main</b> |
| stored in this project. Although the program is currently geared to generating |
| results for the Eclipse Platform builds, the program may work for you given the |
| following conditions:</p><ul> <li>The "eclipse.perf.config" system property |
| used to run tests used three keys config, build and jvm <ul> <li>for example, |
| "eclipse.perf.config=config=relengbuildwin2;build=I20050219-1500;jvm=sun1.4.2_06". |
| <br> <br> </li></ul></li><li>Values for the "build" key begin with either an 'I' |
| or 'N' with the exception of the reference builds (for example, 3.0). <ul> <li>The |
| line graphs plot the reference build, all builds starting with 'I' and the seven |
| 'N' builds preceding the current build.</li><li>This can be overriden with the use of the "-current.prefix" parameter where you can specify a comma-separated list of prefixes for build ids to include in the main line graph.<br> <br> </li></ul></li><li>Scenario names |
| all contain a prefix "<component>.test" <ul> <li>For example, |
| it is assumed that the scenario "org.eclipse.help.tests.performance.appserver.TestAppserver#testAppserverCycle()" |
| belongs to the "org.eclipse.help" component. </li><li>Unique scenario |
| prefixes are derived from all the scenarios in the database and are used to group |
| results.</li></ul></li></ul><p><br> For purposes of this document, the term "<a href="images/descriptions.html#fp">fingerprint</a>" |
| will be used to refer to a bar graph which represents the performance summary |
| at a global or component level for each test configuration.<br> <br> The term |
| "<a href="images/descriptions.html#sst">scenario status table</a>" will |
| be used to refer to a table of scenarios with green checks and red x indicators |
| below a component fingerprint. This table gives an overall view of the status |
| for all scenarios for a component for all configurations.<br> </p><p>The term |
| "<a href="images/descriptions.html#sr">scenario results</a>" will be |
| used to refer to a web page comparing current performance measurements to the |
| baseline measurements for one scenario on one configuration. The web page displays |
| the raw data for the current and baseline measurements and also displays line |
| graphs showing the measured values over the course of builds between the current |
| and baseline.</p><p><br> <b>Requirements</b> </p><ul> <li>Windows or Linux (x86) |
| operating system</li><li>Cloudscape or Derby plug-in</li><li>org.eclipse.releng.basebuilder</li><li>1.4 |
| level jvm installed and on path</li></ul><p><b>Setup</b><br> </p><ul> <li>Checkout |
| org.eclipse.releng.basebuilder from cvs repository dev.eclipse.org:/home/eclipse |
| (HEAD should be fine).</li><li>Create a Cloudscape or derby plug-in as described |
| <A HREF="http://dev.eclipse.org/viewcvs/index.cgi/*checkout*/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html?rev=HEAD&content-type=text/html#cloudscape">here</A>.</li><li>Set |
| the environment variable "dbloc" to the location of the database containing |
| the performance data<br> For example:<br> <ul> <li> export dbloc=net://trelenggtk.ottawa.ibm.com |
| (Linux)</li><li>set dbloc==net://trelenggtk.ottawa.ibm.com (Windows)<br> </li></ul></li></ul><p><b>Running |
| the program</b></p><p>From a terminal or dos prompt,<br> </p><OL><LI>Set the environment |
| variable "dbloc" to the location of the database containing the performance |
| data<BR> For example:<UL><LI>export dbloc=net://trelenggtk.ottawa.ibm.com (Linux)</LI><LI>set |
| dbloc==net://trelenggtk.ottawa.ibm.com (Windows)<BR><BR></LI></UL></LI><LI>cd |
| to org.eclipse.releng.basebuilder\plugins\org.eclipse.test.performance.ui\scripts<BR></LI><LI>If |
| running on Linux, execute "chmod 755 genresults"</LI><LI>Execute the |
| following command:<b><br> <br> genresults -baseline <baseline build id> |
| -current <current build id> -config <comma-separated list of configs> |
| -jvm <jvm name> -output <output dir> [-config.properties <properties>] |
| [highlight <buildid patterns>] [scenario.pattern <scenario prefix |
| patterns>] [-fingerprints][-scenarioresults][-current.prefix <comma-separated list of build id prefixes>][-baseline.prefix <comma-separated list of build id prefixes>]</b></LI></OL><br> Parameter descriptions: |
| <blockquote> <p>-baseline <baseline build id. A previous value used in the |
| "build" key in the eclipse.perf.config system property or the eclipse.perf.assertAgainst |
| property. > <br> -current <current build id. The value used in the "build" |
| key in the eclipse.perf.config system property.><br> -config <a comma-separated |
| list of names of configurations for which to produce results. The values used |
| should be the same as values specified for the "config" key in the eclipse.perf.config |
| system property.><br> -jvm <jvm description. The value used in the "jvm" |
| key in the eclipse.perf.config system property.><br> -output <path to an |
| output directory for fingerprints and scenarios for each configuration specified |
| in -config parameter. Fingerprint gifs and html files are generated in the directory |
| specified, configuration specific subdirectories are created to store scenario |
| results and line graphs.></p></blockquote><p><br> Optional arguments:</p><blockquote> |
| <p>-config.properties <semi-colon separated list of: config, alternate config |
| description. The value should be quoted if there are spaces in the value.><BR>of builds which match any prefix in this list. When omitted, a magenta line is draw on the graph which represents the baseline value specified in the -baseline parameter.><br> -highlight <Comma-separated |
| list of build Id prefixes(excluding 'I' and 'N'), which are used to add and highlight |
| the most recent matching buildId on line graphs."><br> -scenario.pattern |
| <Scenario prefix pattern used to query performance results database. Can be |
| used to generate results for a single component or subset of scenarios.><br> |
| -fingerprints or -scenarioresults <use one or the other to generate fingerprints |
| or scenario results only. Not specifying either will execute both.><BR>-baseline.prefix <semi-colon separated list of: build id prefixes used in eclipse.perf.assertAgainst property or eclipse.perf.config system property. Values used to generate a second line graph representing changes in repeated baseline test runs>.<BR>-current.prefix <semi-colon separated list of: build id prefixes used in eclipse.perf.config system property. Values here used to override default of "N,I". Used to select build id's to include in main line graph.></p></blockquote> |
| </html> |
