| <!doctype html public "-//w3c//dtd html 4.0 transitional//en"> |
| <html> |
| <head> |
| <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> |
| <meta name="Author" content="Build"> |
| <meta name="GENERATOR" content="Mozilla/4.5 [en] (WinNT; I) [Netscape]"> |
| <title>The Workbench Test Framework</title> |
| </head> |
| <body> |
| |
| <h2> |
| The Workbench Test Suite</h2> |
| Dave Springgay |
| <br>September 2001 |
| <h3> |
| In Brief</h3> |
| This document describes the UI Test Suite. This test suite is available |
| from <tt>org.eclipse.ui.tests</tt> in the desktop repository and consist |
| of a series of non-interactive sniff tests which are run after a build, |
| plus a set of interactive tests which require human feedback. You |
| can run any of these from the JUnit Eclipse Plugin Tests launcher or from |
| the command line. |
| <p>The <tt>org.eclipse.ui.tests</tt> project is organized in hierarchical |
| fashion. The root is <tt>UiTestSuite</tt> in <tt>org.eclipse.ui.tests</tt>. |
| This test suite contains a collection of "specialized suites" which test |
| specific areas of functionality or components in the workbench. Currently |
| there is only one suite, <tt>ApiTestSuite |
| </tt>in<tt> org.eclipse.ui.tests.api</tt>, |
| which contains a number of test cases which deal specifically with workbench |
| API. A <tt>uiTest</tt> extension is declared for each suite so that |
| you can run every test in the workbench or a subset of the tests. |
| <p>To run the UiTestSuite from within Eclipse just select UiTestSuite.java |
| in the navigator and invoke Run > JUnit Eclipse Plugin Tests. The |
| results of the test suite will appear in the JUnit view. |
| <h3> |
| Automated Sniff Tests</h3> |
| The following process is used to run the workbench UI sniff tests after |
| a build. |
| <p>To create the sniff test files .. |
| <ol> |
| <li> |
| Export the 4 test projects into a zip file. The options in the export |
| wizard should be set to "Compress the contents of the file" and not "Create |
| directory Structure".</li> |
| </ol> |
| To run the sniff tests in a fresh build of Eclipse .. |
| <ol> |
| <li> |
| Unzip the zip file in the Eclipse plugins directory.</li> |
| |
| <li> |
| On Windows copy uiSniff.bat in the org.eclipse.ui.tests plugin directory |
| to the Eclipse home directory. On Linux copy uiSniff instead.</li> |
| |
| <li> |
| Run uiSniff.bat ( or uiSniff ). Redirect the results into a file.</li> |
| </ol> |
| |
| <h3> |
| Test Patterns</h3> |
| In our own testing we have discovered a number of common test patterns. |
| In order to encourage a consistent approach for testing they are published |
| below. |
| <p><b>Simple Setters, Getters and Commands</b> |
| <p>Scenario: You want to test setters, commands and getters on an interface. |
| <p>Method: Call the setter or command which affects the object state. |
| Call the getter to verify that state. |
| <p><b>Top Down Coverage</b> |
| <p>Scenario: You want to demonstrate thorough coverage of a component. |
| <p>Method: Start at the object root of the instance hierarchy and test |
| every class downwards to the leaf classes. Stop when you encounter |
| a layer which already has JUnit test cases. For instance, in the |
| workbench UI don't test JFace, SWT, or core. They should have their |
| own test suites and are assumed to work. |
| <p><b>Superclass Subclass</b> |
| <p>Scenario: You want to test D, which is a subclass of B. |
| <p>Method: Implement a test case for B called BTest. Then create |
| a subclass of BTest called DTest which tests D. If B is an abstract |
| class use a factory method in BTest to create the test object. |
| <p><b>Listener Source</b> |
| <p>Scenario: You want to test S, a class which fires events when a particular |
| situation occurs. |
| <p>Method: Implement a listener for S which records the reception of events. |
| Then write a test class for S called STest which does something which should |
| cause those events to fire. Verify the reception of events afterwards. |
| <p><b>Multiple Session</b> |
| <p>Scenario: You want to test the persistance of state from one session |
| to the next. |
| <p>Method: You need to create two test suites. One test suite will |
| set up the state. The other will verify the state. Run them |
| sequentially in two separate processes. |
| <p><b>Global State</b> |
| <p>Scenario: In Eclipse the workbench is a global object. Unfortunately, |
| this means that one test case may modify that state and affect the outcome |
| of another unrelated test case. How can you avoid this problem? |
| <p>Method: If the test case modifies the state of a window or something |
| in the window you should create a new window as part of the setUp for the |
| test case. Run the test code within that window and then close the |
| test window in the tearDown method. The modified state will be discarded |
| when the window is closed |
| <p><b>Lifecycle</b> |
| <p>Scenario: Within the workbench there are various interfaces, such as |
| IViewPart, which are defined as API and implemented by plugin code. |
| There is no need to test the implementation of the interface, but it is |
| good to test the lifecycle as implemented by objects which call the interface. |
| <p>Method: Define a class X which implements the interface and records |
| the invocation of various methods. Create a scenario where this class |
| is loaded and should receive events. Afterwards, test that those |
| methods were called. |
| <p><b>Piercing the Encapsulation</b> |
| <p>Scenario: To test the behavior of commands which modify the state of |
| the object when there are no public interfaces to query that state. |
| <p>Method: If possible, cast the interface to a concrete class with additional |
| public methods. For instance, in the workbench the underlying structure |
| for IWorkbench is exposed in Workbench. Given a Workbench object, |
| you can get the menu, toolbar, etc, and interact directly with the actions |
| to verify their state or invoke them directly. This is also a useful |
| way to simulate action invocation from the UI. |
| <p><b>Mock Objects</b> |
| <p>Scenario: To test the implementation of an interface which instantiates |
| an extension. For instance, in the test case for IWorkbenchPage we |
| need to open views and editors. |
| <p>Method: If we reference views and editors which exist within the Workbench |
| UI Standard Components project the test case is vulnerable to change in |
| those components. In this case we're not testing the components, |
| we're testing IWorkbenchPage, so we implemented some light weight mock |
| views and editors which do nothing more than record their own creation |
| and lifecycle. |
| <br> |
| <br> |
| <br> |
| </body> |
| </html> |