| <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> |
| <html lang="en"> |
| <head> |
| <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> |
| <title>org.eclipse.test.performance.ui readme</title> |
| </head> |
| |
| <p> |
| <b><font size="+2">README</font></b><br> |
| </p> |
| <p> |
| Last updated: October 26, 2005<br> |
| </p> |
| <p> |
| <b>Introduction</b><br> <br> This readme describes how to |
| create performance charts and tables similar to those available on the |
| eclipse platform download pages using the stand-alone java program <b>org.eclipse.test.performance.ui.Main</b> |
| stored in this project. Although the program is currently geared to |
| generating results for the Eclipse Platform builds, the program may |
| work for you given the following conditions: |
| </p> |
| <ul> |
| <li>The "eclipse.perf.config" system property used to |
| run tests used three keys config, build and jvm |
| <ul> |
| <li>for example, |
| "eclipse.perf.config=config=relengbuildwin2;build=I20050219-1500;jvm=sun1.4.2_06". |
| <br> <br> |
| </li> |
| </ul> |
| </li> |
| <li>Values for the "build" key begin with either an 'I' or 'N' |
| with the exception of the reference builds (for example, 3.0). |
| <ul> |
| <li>The line graphs plot the reference build, all builds |
| starting with 'I' and the seven 'N' builds preceding the current |
| build.</li> |
| <li>This can be overriden with the use of the |
| "-current.prefix" parameter where you can specify a |
| comma-separated list of prefixes for build ids to include in the |
| main line graph.<br> <br> |
| </li> |
| </ul> |
| </li> |
| <li>Scenario names all contain a prefix |
| "<component>.test" |
| <ul> |
| <li>For example, it is assumed that the scenario |
| "org.eclipse.help.tests.performance.appserver.TestAppserver#testAppserverCycle()" |
| belongs to the "org.eclipse.help" component.</li> |
| <li>Unique scenario prefixes are derived from all the scenarios |
| in the database and are used to group results.</li> |
| </ul> |
| </li> |
| </ul> |
| <p> |
| <br> For purposes of this document, the term "<a |
| href="images/descriptions.html#fp">fingerprint</a>" will be used |
| to refer to a bar graph which represents the performance summary at a |
| global or component level for each test configuration.<br> <br> |
| The term "<a href="images/descriptions.html#sst">scenario |
| status table</a>" will be used to refer to a table of scenarios with |
| green checks and red x indicators below a component fingerprint. This |
| table gives an overall view of the status for all scenarios for a |
| component for all configurations.<br> |
| </p> |
| <p> |
| The term "<a href="images/descriptions.html#sr">scenario |
| results</a>" will be used to refer to a web page comparing current |
| performance measurements to the baseline measurements for one scenario |
| on one configuration. The web page displays the raw data for the |
| current and baseline measurements and also displays line graphs showing |
| the measured values over the course of builds between the current and |
| baseline. |
| </p> |
| <p> |
| <br> <b>Requirements</b> |
| </p> |
| <ul> |
| <li>Windows or Linux (x86) operating system</li> |
| <li>org.eclipse.releng.basebuilder</li> |
| <li>JVM 11 installed and on path</li> |
| </ul> |
| <p> |
| <b>Running the program</b> |
| </p> |
| <p> |
| From a terminal or dos prompt,<br> |
| </p> |
| <OL> |
| <LI>cd to |
| org.eclipse.releng.basebuilder\plugins\org.eclipse.test.performance.ui\scripts<BR> |
| </LI> |
| <LI>If running on Linux, execute "chmod 755 genresults"</LI> |
| <LI>Execute the following command:<b><br> <br> |
| genresults -baseline <baseline build id> -current <current |
| build id> -config <comma-separated list of configs> -jvm |
| <jvm name> -output <output dir> [-config.properties |
| <properties>] [highlight <buildid patterns>] |
| [scenario.pattern <scenario prefix patterns>] |
| [-fingerprints][-scenarioresults][-current.prefix <comma-separated |
| list of build id prefixes>][-baseline.prefix <comma-separated |
| list of build id prefixes>]</b></LI> |
| </OL> |
| <p> |
| <br> Parameter descriptions: |
| </p> |
| <blockquote> |
| <ul> |
| <li>-baseline - baseline build id. A previous value used in the |
| "build" key in the eclipse.perf.config system property or |
| the eclipse.perf.assertAgainst property.</li> |
| <li>-current - current build id. The value used in the |
| "build" key in the eclipse.perf.config system property.</li> |
| <li>-config - a comma-separated list of names of configurations |
| for which to produce results. The values used should be the same as |
| values specified for the "config" key in the |
| eclipse.perf.config system property.</li> |
| <li>-jvm -jvm |
| description. The value used in the "jvm" key in the |
| eclipse.perf.config system property. |
| </li> |
| <li>-output - path to an output directory for fingerprints and |
| scenarios for each configuration specified in -config parameter. |
| Fingerprint gifs and html files are generated in the directory |
| specified, configuration specific subdirectories are created to store |
| scenario results and line graphs.</li> |
| </ul> |
| </blockquote> |
| <p> |
| <br> Optional arguments: |
| </p> |
| <blockquote> |
| <ul> |
| <li>-config.properties - semi-colon separated list of: config, |
| alternate config description. The value should be quoted if there are |
| spaces in the value.<BR>of builds which match any prefix in this |
| list. When omitted, a magenta line is draw on the graph which |
| represents the baseline value specified in the -baseline parameter. |
| </li> |
| <li>-highlight - Comma-separated list of build Id |
| prefixes(excluding 'I' and 'N'), which are used to add and highlight |
| the most recent matching buildId on line graphs.</li> |
| <li>-scenario.pattern - Scenario prefix pattern used to query |
| performance results database. Can be used to generate results for a |
| single component or subset of scenarios.</li> |
| <li>-fingerprints or -scenarioresults - use one or the other to |
| generate fingerprints or scenario results only. Not specifying either |
| will execute both.</li> |
| <li>-baseline.prefix - semi-colon separated list of: build id |
| prefixes used in eclipse.perf.assertAgainst property or |
| eclipse.perf.config system property. Values used to generate a second |
| line graph representing changes in repeated baseline test run.</li> |
| <li>-current.prefix - semi-colon separated list of: build id |
| prefixes used in eclipse.perf.config system property. Values here |
| used to override default of "N,I". Used to select build |
| id's to include in main line graph.</li> |
| </ul> |
| </blockquote> |
| </html> |