| <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> |
| <html> |
| <head> |
| <title>Session Test Framework</title> |
| <link rel="stylesheet" href="../default_style.css" type="text/css"> |
| </head> |
| <body text="#000000" bgcolor="#ffffff"> |
| <h1>Session Test Framework</h1> |
| <blockquote> |
| <p><b>Summary</b><br> |
| Session tests in Eclipse are tests that require that a new Eclipse session |
| be launched for every test case run. Thus, they have additional requirements |
| regarding controling the environment test cases are run (VM, set of plug-ins |
| available, configuration, instance location, command line parameters) and |
| how results are communicated back to the test runner. This FAQ-like document describes |
| how the Platform/Core implemented support for running session tests in the |
| hopes of encouraging adoption of this framework not only by all the Platform/Core |
| team members but also by members of other Eclipse teams interested in developing |
| their own session tests. </p> |
| <p>Last Modified: December 22nd, 2004</p> |
| </blockquote> |
| <ol> |
| <li> |
| <p><strong>What is a session test?</strong></p> |
| <p>It is a test case that needs to be run isolated in a separate VM instance. |
| Technical details: The VM running a set of test cases (the <em>controller |
| VM</em>) will spawn a second instance (the <em>target VM</em>) for each |
| session test case to be run. The test results are collected (by using a |
| socket) and presented to the test runner as if the test had run in the same |
| VM.</p> |
| </li> |
| <li> |
| <p><strong>Where can I get the framework?</strong></p> |
| <p> Check out the <code>org.eclipse.core.tests.harness</code> project from |
| dev.eclipse.org. The infrastructure for session tests is in the <code>org.eclipse.core.tests.session</code> |
| package. It does not hurt mentioning that besides requiring this plug-in, |
| you should have <code>org.junit</code> as a pre-requisite as well.</p> |
| </li> |
| <li> |
| <p><strong>What it is the easiest way to transform my regular plug-in tests |
| into session tests?</strong></p> |
| <p> Change the test suite creation to use <code>org.eclipse.core.tests.session.SessionTestSuite</code> |
| instead of <code>junit.framework.TestSuite</code>. Like this:</p> |
| <p>Before:</p> |
| <pre> |
| public class MyTestCase extends TestCase { |
| ... |
| public void test1() { |
| ... |
| } |
| ... |
| public static Test suite() { |
| return new TestSuite(MyTestCase.class); |
| } |
| } |
| </pre> |
| <p>After:</p> |
| <pre> |
| public class MyTestCase extends TestCase { |
| ... |
| public void test1() { |
| ... |
| } |
| ... |
| public static Test suite() { |
| return new SessionTestSuite("org.myplugin.tests", MyTestCase.class); |
| } |
| } |
| </pre> |
| <p>The plug-in id is necessary so the plug-in which the class containing the |
| test case should be loaded from can be determined. Note hat it is possible |
| to build an empty suite and then manually add test cases and/or test suites. |
| When running your tests again, every test case in the suite (included those |
| in nested test suites) will be run in a separate Eclipse session. Note also |
| that if a session test suite contains another session test suite (or any |
| other special types of test suite), the outer session test suite will rule.</p> |
| </li> |
| <li><strong>Why do I get a <em>"...SetupException: No setup descriptions |
| found. Ensure you are specifying the path for an existing setup file...</em>"?</strong> |
| <p> You are running your session test suite as a regular <em>JUnit test</em> |
| instead of as a <em>Plug-in JUnit test</em>. A default setup (in other words, |
| command line) for launching new Eclipse sessions when running session tests |
| can be computed when running the test suite as a plug-in test, so no further |
| configuration is necessary. It basically contains the same parameters as |
| the ones used to launch the controller instance, with a different instance |
| location (the -data parameter). When launching the controller instance as |
| a regular JUnit test suite, the setup for running session tests has to be |
| configured manually. The framework looks for a <code>default-setup.xml</code> |
| file in the current directory, but a different file name and location can |
| be specified by using the <code>setup.file</code> system property. If not |
| running in Eclipse, and no setup file can be found, the execution fails |
| with this exception. See also next FAQ.</p> |
| </li> |
| <li> |
| <p><strong>Should I run my tests using a JUnit or PDE Junit launch configuration?</strong></p> |
| <p>Running with a PDE Junit launch configuration will allow you to test code |
| in your workspace. Also, launching with PDE JUnit frees you from having |
| to provide configuration parameters, since reasonable defaults will be obtained |
| from the controlling Eclipse. If you are still interested in running your |
| tests as a JUnit launch configuration, see the next FAQ.</p> |
| </li> |
| <li> |
| <p><strong>How do I run session tests from a JUnit (not PDE JUnit) launch |
| configuration?</strong> </p> |
| <p>Ensure you provide a valid set of setup files. Setup files describe option |
| sets that can be used when launching Eclipse. See question about options |
| supported by the framework.</p> |
| </li> |
| <li> |
| <p><strong>How do I run UI-oriented tests?</strong> </p> |
| <p>Make sure you also set the application when launching the target instance |
| to be <code>SessionTestSuite.UI_TEST_APPLICATION</code>:</p> |
| <pre>SessionTestSuite suite = new SessionTestSuite("org.myplugin.tests", MyTestCase.class); |
| suite.setApplicationId(SessionTestSuite.UI_TEST_APPLICATION);</pre> |
| </li> |
| <li><strong>How can I see the command-line used for the Eclipse instance where my session |
| test runs?</strong> |
| <p>Use the <code>setup.debug</code> boolean system property (check the question about supported options).</p> |
| </li> |
| <li><strong>How do I debug a failing test case?</strong> |
| <p><strong>The easy way:</strong> run the failing test case in isolation as |
| a regular JUnit plug-in test. Ensure the command-line matches the one build |
| by the session test framework when running it as a session test.</p> |
| <p><strong>The hard way:</strong> use the <code>remote_debug</code> setup |
| option (see the question on the supported options in this document). This |
| is not much fun. Every single test case will be launched in a separate VM |
| in remote debug mode. Thus, it will block, waiting for a client VM to connect. |
| You will have to use a Remote Application Debug launch configuration for |
| every test case to be run. There is no way to run only some of the test |
| cases in a session suite in remote debug mode.</p> |
| </li> |
| <li><strong>How do I run performance session tests?</strong> |
| <p>Use the <code>PerformanceSessionTestSuite</code> instead.</p> |
| </li> |
| <li><strong>How do I run session tests that have private instance locations?</strong> |
| <p>Use the <code>WorkspaceSessionTestSuite</code> instead. Every test suite |
| has its own instance location (workspace), all test cases are run on separate |
| sessions on the same workspace. After the last test case is run, that location |
| is deleted.</p> |
| </li> |
| <li><strong>I just want to try out some examples. Any pointers?</strong> |
| <p>The <code>org.eclipse.core.tests.session.samples</code> package has some |
| examples. If you load the org.eclipse.core.tests.runtime and org.eclipse.core.tests.resources |
| projects, you will find real-world examples of how to use this framework. |
| Just look for the subclasses of <code>SessionTestSuite</code>.</p> |
| </li> |
| <li><strong>What are all supported options and how to use them?</strong> |
| <p>The options for the session test framework are specified in the form of |
| system properties. <em>You only have to use these options if running with |
| the default settings (the currently target JRE and Eclipse and default command |
| line options) are not appropriate.</em></p> |
| <ul> |
| <li><strong>setup.options</strong> - only valid when setup files are available. |
| Setup files describe option sets, which are collections of command-line |
| options that can be used while running Eclipse. Examples: |
| <p><code>-Dsetup.options=sun_1.5,eclipse_3.1_M1,big_memory</code></p> |
| <p>would run Eclipse 3.1 M1 using Sun JRE 1.5 with whatever amount of |
| memory the <code>big_memory</code> option set specifies. While:</p> |
| <p><code>-Dsetup.options=ibm_1.4,eclipse_3.0,no_jit</code></p> |
| <p>would run Eclipse 3.0 on IBM JRE 1.4 with Just-in-Time compilation |
| disabled</p> |
| <p>The meaning of every option set must be specified in setup files (see |
| below).</p> |
| </li> |
| <li><strong>setup.files</strong> - setup files provide option set definitions |
| for running the session tests. Setup files can be chained, so a more general |
| setup file (used by the whole team) can be customized with a user specific |
| setup file. This is useful since option sets must specify file system |
| locations, and every user has its own layout for JRE and Eclipse installs. |
| The expected use of this option is: |
| <p><code>-Dsetup.files=/tmp/default-setup.xml,~/user-setup.xml</code></p> |
| <p>Here, the <code>default-setup.xml</code> file is shared by the whole |
| team and defines all option sets that the team may use. The <code>user-setup.xml</code> |
| merely overrides location sensitive options to adapt them to the user |
| specific file system layout. Look <a href="./default-setup.xml">here</a> |
| for an example of a default setup file and <a href="./user-setup.xml">here</a> |
| for an example of a user custom setup file. These examples are intended |
| to be self-explanatory and are actually useful as they are (the user-specifc |
| setup file has to be tweaked to support your specific file system layout.</p> |
| </li> |
| <li><strong>setup.override.eclipseArgs</strong> - allows the user to manually |
| override command-line options corresponding to Eclipse arguments. Example: |
| <p><code>-Dsetup.override.eclipseArgs=data=c:/workspace;debug;showlocation</code></p> |
| </li> |
| <li><strong>setup.override.vmArgs</strong> - allows the user to manually |
| override command-line options corresponding to VM arguments (other than |
| system properties). Example: |
| <p><code>-Dsetup.override.vmArgs=Xint;Xmx256m;hotspot;cp=startup.jar</code></p> |
| </li> |
| <li><strong>setup.override.systemProperties</strong> - allows the user to |
| manually override command-line options corresponding to system properties. |
| Example: |
| <p><code>-Dsetup.override.systemProperties=osgi.checkConfiguration;osgi.locking=none</code></p> |
| </li> |
| <li><strong>setup.debug</strong> - enables the session test framework debug |
| mode. In this mode, among other things, the command-line for every session |
| launched is dumped to the console, so it is useful to verify which option |
| sets are being selected and how they map to the command line.</li> |
| </ul> |
| <p></p> |
| </li> |
| <li><strong>What if I find problems using the session test framework or would |
| like to request/propose an enhancement?</strong> |
| <p>Please report them in Bugzilla against the Platform/Runtime component.</p> |
| </li> |
| </ol> |
| </body> |
| </html> |
