blob: 380ffe31004bf11ebcc802317d830c7163df3ddc [file] [log] [blame]
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="GENERATOR" content="Microsoft FrontPage 4.0">
<title>Test Plan - 3.0</title>
<link rel="stylesheet" href="http://dev.eclipse.org/default_style.css" type="text/css">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<h1><a name="top"></a>3.0 Core Test Plan</h1>
<p>Testing will cover the functional areas on these platforms/VMs</p>
<table border="1" width="63%">
<tr>
<td width="54%">&nbsp;</td>
<td width="23%">JDK 1.4.2</td>
<td width="23%">SC 1.4.2</td>
</tr>
<tr>
<td>Windows 2000 (FAT32)</td>
<td>&nbsp;</td>
<td>&nbsp;</td>
</tr>
<tr>
<td>Windows XP (NTFS)</td>
<td>&nbsp;</td>
<td>&nbsp;</td>
</tr>
<tr>
<td>Mac OSX</td>
<td>&nbsp;</td>
<td>N/A</td>
</tr>
<tr>
<td>Linux</td>
<td>&nbsp;</td>
<td>&nbsp;</td>
</tr>
</table>
<ul>
<li><a href="#launcher">Launcher</a></li>
<li><a href="#osgi">OSGi</a></li>
<li><a href="#equinox">Runtime - Equinox</a></li>
<li><a href="#jobs">Runtime - Jobs</a></li>
<li><a href="#preferences">Runtime - Preferences</a></li>
<li><a href="#resources">Resources</a></li>
<li><a href="#pde-build">PDE Build</a></li>
<li><a href="#performance">Performance</a></li>
</ul>
<p>Points to remember when testing</p>
<ul>
<li>File a bug when something does not work. Do not try to fix the problem
unless it prohibits further testing.</li>
<li>When
a test can be automated and is not part of our test suite, add a JUnit
test to the test suite.</li>
</ul>
<hr>
<h3><a name="launcher"/>Launcher</h3>
<h4>No config.ini</h4>
<ul>
<li>install Eclipse</li>
<li>delete the configuration dir</li>
<li>start Eclipse</li>
<li>confirm that everything goes smoothly. The only thing that should be different
is the lack of splash screen.</li>
<li>Note: There may be an issue with the eclipse.product not being set and the
runtime not knowing which application to run. We may no longer be able
to run without a config.ini. If there is a strong usecase for this then
we could embed a valud in eclipse.properties if we had to.</li>
</ul>
<h4>Exit data</h4>
<ul>
<li>set the exit data system property to a string value</li>
<li>exit Eclipse with the an error code other than 0, 23 or 24 and ensure
the exit data string is properly displayed</li>
</ul>
<h4>Restart</h4>
<ul>
<li>start Eclipse, use the VM, commands and vmargs System properties to construct
a new command line</li>
<li>set this new command line in the exitdata</li>
<li>exit Eclipse with return code 24</li>
<li>confirm that the specified command was run on restart</li>
</ul>
<a href="#top">Back to top</a><br><hr>
<h3><a name="osgi"/>OSGi</h3>
<h4>Command line options</h4>
<ul>
<li>try various command line options and ensure that the corresponding system
property values are set. In particular,
<ul>
<li>-configuration</li>
<li>-product</li>
<li>-application</li>
<li>-install</li>
<li>-data</li>
</ul>
</li>
</ul>
<h4>Shared Install 1</h4>
<ul>
<li>install Eclipse on shared drive</li>
<li>run once with -initialize, mark read-only</li>
<li>run one instance, exit</li>
<li>run another instance (as different user or on different machine), exit</li>
<li>run two instances at the same time (as different users or on different
machines)</li>
<li>exit in same order as start (i.e., start 1, start 2, exit 1, exit 2)</li>
</ul>
<h4>Shared Install 2</h4>
<ul>
<li>install Eclipse on shared drive</li>
<li>run once with -initialize, mark read-only</li>
<li>run user instance, exit</li>
<li>start admin instance (with read/write permissions)</li>
<li>install new features using update manager, exit</li>
<li>restart previous user instance</li>
<li>confirm that newly installed features are present and funcitonal</li>
<li>check that no additional files have been written in the local configuration
area</li>
</ul>
<h4>Shared Install 3</h4>
<ul>
<li>install Eclipse on shared drive</li>
<li>run once with -initialize, mark read-only</li>
<li>run one instance</li>
<li>install new features using update manager</li>
<li>restart. New features should be there and totally functional</li>
<li>start another instance as the same user on the same machine</li>
<li>previously installed features should be present and functional</li>
<li>start another instance on a different machine or as a different user</li>
<li>previoulsy installed features should NOT be present </li>
</ul>
<h4>Resolution</h4>
<ul>
<li>confirm the new range-based version matching works as expected, looking
for corner cases</li>
<li>check resolution in the presence of fragments</li>
</ul>
<p>Note: should consider reenabling the old resolver tests, doing the necessary
port to work with the new APIs</p>
<a href="#top">Back to top</a>
<hr>
<h3><a name="equinox"/>Runtime - Equinox</h3>
<h4>JAR'd plugins</h4>
<ul>
<li>populate an update site with the Core Tools feature and plugins/fragments.
mark the relevant feature entries as unpack=false.</li>
<li>start Eclipse and install the Core Tools feature using update manager</li>
<li>check that the plugins and fragments did not get exploded</li>
<li>confirm that the Core Tools function as expected</li>
</ul>
<h4>Extension registry</h4>
<ul>
<li>install a plug-in dynamically and check whether its extensions and extension
points are properly added to the registry</li>
<li>remove it and ensure its extensions and extension points go away</li>
<li>do the same as above with two plugins where one of them requires the other.</li>
<li>do the same as above with a plugin and a fragment, where both contribute extensions/extension points.</li>
<li>ensure plugin manifests with valid XML but missing required elements/attributes
in extensions/extension points markup are properly ignored/handled</li>
</ul>
<p><strong>Note</strong>: use the <code>org.eclipse.core.runtime/registry/debug/events/extension</code> debug option
to monitor registry change events</p>
<h4>Nested jars</h4>
<ul>
<li>Zip the Junit folder such that the files are at the root of the archive, and rename the archive to have .jar as extension</li>
<li>Delete the JUnit folder</li>
<li>Start eclipse</li>
<li>In the console type
<pre>
install reference:file:/<junit.jar>
refresh <installNumber of the installed plugin>
ss
</pre>
Check that the junit plugin is installed and resolved.
Run a Junit test.
</ul>
<h4>Code in folders</h4>
<ul>
<li>Unzip the content of the original JUnit.jar into a folder named "code"</li>
<li>In the plugin.xml replace &gt;library name="junit.jar"&lt; by &gt;library name="code/"&lt;</li>
<li>Start eclipse</li>
<li>In the console type
<pre>
install reference:file:/<junit folder>
refresh <installNumber of the installed plugin>
ss
</pre>
Check that the junit plugin is installed and resolved.
Run a Junit test.
</ul>
<h4>Folders in a Jar</h4>
<ul>
<li>Zip the content of the Junit folder from the previous test in such a way that plugin.xml is that the root of the archive</li>
<li>Start eclipse</li>
<li>In the console type
<pre>
install reference:file:/<junit folder>
refresh <installNumber of the installed plugin>
ss
</pre>
Check that the junit plugin is installed and resolved.
Run a Junit test.
</ul>
<a href="#top">Back to top</a><br><hr>
<h3><a name="jobs"/>Runtime - Jobs</h3>
<h4>Test concurrent operations in UI thread</h4>
<ul>
<li>Enable <b>Always run in background</b> preference </li>
<li>Start checkout of platform-core module from CVS</li>
<li>During checkout, switch to the Java perspective</li>
<li>UI may become blocked, but should eventually become responsive</li>
<li>Open Path.java in Java editor</li>
<li>Switch back to CVS perspective</li>
<li>Start checkout of org.eclipse.jdt.ui.tests.refactoring (or other big project)</li>
<li>Open Navigator: New files should start appearing</li>
<li>Add breakpoint to a method in Path.java: should appear quickly</li>
<li>Remove breakpoint: should be responsive</li>
<li>Start typing in Path.java: Should not block</li>
<li>Save Path.java: Should not block</li>
<li>Start typing in Path.java again: Should not block</li>
<li>Revert file: Should not block</li>
</ul>
<h4>Many background tasks</h4>
<ul>
<li>Start CVS checkout of ten or more different projects</li>
<li>Open progress view (double-click in lower right hand corner of workbench window)</li>
<li>All checkout tasks should be making progress</li>
<li>Canceling the checkouts should respond in a timely manner.</li>
</ul>
<h4>Nested blocking acquires</h4>
<ul>
<li>Install the org.eclipse.ui.examples.job plugin</li>
<li>Load org.eclipse.ui.examples.job</li>
<li>Open the Job Factory view</li>
<li>Check "Lock the workspace"</li>
<li>Click "Create Jobs"</li>
<li>Change quantity to 3 and duration to 10 seconds</li>
<li>Click "Touch Workspace"</li>
<li>Cancel the Test Job</li>
<li>Dialog should close</li>
<li>Repeat all above steps except cancel the blocked operation instead of the test job (three times)</li>
<li>Dialog should close but test job should continue running</li>
</ul>
<h4>Shutdown while job is running</h4>
<ul>
<li>Start a long CVS checkout</li>
<li><b>File &gt; Exit</b>.</li>
<li>A progress dialog will appear indicating that the operation is waiting.</li>
<li>Expand <b>Details</b> area and cancel the CVS checkout</li>
<li>Exit should now complete in a timely manner</li>
</ul>
<a href="#top">Back to top</a><br><hr>
<h3><a name="preferences"/>Runtime - Preferences</h3>
<h4>Automatic Migration: 2.1 to 3.0</h4>
<ul>
<li>Start with a 2.1 workspace</li>
<li>Change some settings.</li>
<li>Run 3.0 on that workspace.</li>
<li>Validate preferences.</li>
</ul>
<h4>Explicit Migration: 2.1 to 3.0</h4>
<ul>
<li>Start 2.1.</li>
<li>Change settings.</li>
<li>Export settings.</li>
<li>Start 3.0 (new workspace)</li>
<li>Import settings.</li>
<li>Validate preferences.</li>
</ul>
<h4>Import/Export</h4>
<ul>
<li>Test Import/Export mechanism.</li>
<li>Test project version validation code.</li>
</ul>
<h4>Default values</h4>
<ul>
<li>Test old default initialization code.</li>
<li>Test product customization code.</li>
<li>Test new default inititalization extension point.</li>
</ul>
<h4>Project Preferences</h4>
<ul>
<li>Current people using it in SDK are: Character Encoding, PDE runtime workbench output folder exclusion.</li>
<li>Test sharing project prefs in the repository.</li>
<li>Change file in file-system, auto-refresh on -&gt; changes come into workspace</li>
<li>Change in file-system, auto-refresh off -&gt; IFile#setContents should handle things?</li>
</ul>
<h4>Everything Else...</h4>
<ul>
<li>Should not notice any changes in behaviour from 2.1.</li>
</ul>
<a href="#top">Back to top</a><br><hr>
<h3><a name="resources"/>Resources</h3>
<h4>Encoding - Content types</h4>
<ul>
<li>confirm that the right content type is selected when a complex hierarchy
of content types exist (use the resource properties dialog to see the content
type that was chosen):
<ul>
<li>a .xml file is recognized as XML</li>
<li>a .xml file whose root element is &lt;project&gt; is recognized as an
Ant Build Script</li>
<li>a plugin.xml file is recognized as a Plugin Manifest file</li>
<li>a plugin.xml file whose root element is &lt;project&gt; is recognized
as an Ant Build Script</li>
</ul>
</li>
<li>try the same as above with partial/empty XML files - whenever a file has
a .xml extension, it should at least be recognized as XML. If the contents
are complete enough (root element can be parsed), more specific content types
(such as Ant scripts) should be picked.</li>
<li>confirm that the right encoding is picked for XML files containing a &quot;encoding&quot;
attribute or not (default is UTF-8)</li>
<li>this is a list of all content types in the SDK:
<ul>
<li>org.eclipse.core.runtime.text
<ul>
<li>org.eclipse.core.runtime.xml (*.xml, .classpath, .properties)
<ul>
<li>org.eclipse.ant.core.antBuildFile (*.xml with a &lt;project&gt;
root element)</li>
<li>org.eclipse.pde.core.pluginManifest (plugin.xml)</li>
<li>org.eclipse.pde.core.fragmentManifest (fragment.xml)</li>
</ul>
</li>
<li>org.eclipse.jdt.core.javaProperties (*.properties)
<ul>
<li>org.eclipse.pde.core.pluginProperties (plugin.properties)</li>
</ul>
</li>
<li>org.eclipse.jdt.core.JARManifest (MANIFEST.MF)</li>
<li>org.eclipse.jdt.core.javaSource</li>
</ul>
</li>
</ul>
</li>
</ul>
<h4>Encoding - BOMs</h4>
<ul>
<li>confirm text files (or any derived content types such as XML and Java source
files) are said to have the right encoding when they present a Byte Order
Mark (use Notepad to generate those)</li>
<li>confirm opening such files works ok</li>
<li>confirm encoding of files with BOM can be overridden by setting a user-specified
encoding</li>
</ul>
<h4>Encoding - Preferences</h4>
<ul>
<li>check the encoding-related info is properly updated when changes are made
directly to the underlying preferences file (&lt;project&gt;/.settings/org.eclipse.core.resources.prefs)
- try editing the file in a text editor, deleting the file, making the file
out-of-sync</li>
</ul>
<a href="#top">Back to top</a><br><hr>
<h3><a name="pde-build"/>PDE-Build</h3>
<h4>Plugin with "." on the classpath</h4>
<ul>
<li> Using the "new plugin wizard" create a plugin named Dot1 with "." as a
jar name. </li>
<li> Export &quot;deployable plug-in&quot; as a zip. Verify that the classes
are not into a jar. </li>
<li> Export &quot;deployable plug-in&quot; as folder. Verify that the classes
are at the root of the folder.</li>
</ul>
<h4>Compiling against a plugin with "." on the classpath</h4>
<ul>
<li> Create a plugin named Normal</li>
<li> add a dependency to Dot1 in the plugin.xml</li>
<li> add a code dependency from a class in Normal to a class in Dot1</li>
<li> try to export Normal as &quot;deployable plug-in&quot; </li>
<li> try to export Normal and Dot1as &quot;deployable plug-in&quot; </li>
<li> confirm that no compile errors occured (if there were you would get an
error dialog at the end of the export)</li>
</ul>
<h4>Plugin with a folder on the classpath</h4>
<ul>
<li> Create a plugin named Folder1 with "myFolder" as a jar name</li>
<li> edit the build.properties and add a slash after myFolder in bin.includes
property</li>
<li> Export &quot;deployable plug-in&quot; as folder, verify that the class
files are rooted in a folder called myFolder</li>
</ul>
<h4>Testing the build order</h4>
<ul>
<li> in the plugin Folder1, add the following things in the runtime section of the plugin.xml</li>
<pre>
&lt;library name="yourFolder"&gt;
&lt;export name="*"/&gt;
&lt;/library&gt;
&lt;library name="library.jar"&gt;
&lt;export name="*"/&gt;
&lt;/library&gt;
</pre>
<li> replace the content of the build.properties by </li>
<pre>
source.myFolder = src/
output.myFolder = bin/
source.yourFolder = src/
output.yourFolder = bin/
source.library.jar = src/
output.library.jar = bin/
bin.includes = plugin.xml,\
myFolder/,
yourFolder/
jars.compile.order = yourFolder, library.jar, myFolder
</pre>
<li> generate manually the build.xml (select plugin.xml, context menu, PDE Tools
-&gt; Create Ant Build file) and verify that the build.jars target successively
and in this order list yourFolder, library.jar and myFolder</li>
</ul>
<h4>Reuse the plugin Normal</h4>
<ul>
<li> add a dependency to Folder1</li>
<li> add a code dependency from a class in Normal to a class in Folder1</li>
<li> Export Normal as &quot;deployable plug-in&quot; </li>
<li> Export Normal and Folder1 as &quot;deployable plug-in&quot; </li>
</ul>
<h4>Export of qualified plugins</h4>
<ul>
<li> Create a plugin Qualified where the version number ends is 1.0.0.qualifier.</li>
<li> open the build.properties and add
<pre>qualifier = context</pre>
</li>
<li> Export &quot;deployable plug-in&quot; as zip and verify that the name of
the zip has 1.0.0.&lt;date of the day&gt;</li>
<li> Export &quot;deployable plug-in&quot; as folder and verify that the folder
as the name 1.0.0.&lt;date of the day&gt;</li>
<li> Verify that the version number in the plugin.xml has been updated</li>
<br>
<li> change the build.properties to
<pre>qualifier = NONE</pre>
</li>
<li> Export &quot;deployable plug-in&quot; as zip and verify that the name of
the zip is 1.0.0</li>
<li> Export &quot;deployable plug-in&quot; as folder and verify that the name
of the directory is 1.0.0</li>
<br>
<li> change the build.properties to
<pre>qualifier = myValue</pre>
</li>
<li> Export &quot;deployable plug-in&quot; as zip and verify that the name of
the zip is 1.0.0.myValue</li>
<li> Export &quot;deployable plug-in&quot; as folder and verify that the name
of the directory is 1.0.0.myValue</li>
</ul>
<h4>Export of qualified fragments</h4>
<ul>
<li> Create a fragment QualifiedFragment to any plugin, set the version number as 1.0.0.qualifier</li>
<li> repeat the same steps as in the test with plugin</li>
</ul>
<h4>Export of qualified features</h4>
<ul>
<li> Create a feature QualifiedFeature, set the version number as 1.0.0.qualifier (don't need to add a plugin in the feature)</li>
<li> repeat the same steps as in the test with plugin</li>
</ul>
<h4>Export of qualified plugins and features</h4>
<ul>
<li> Take the feature previously created</li>
<li> add the plugin "Qualified" in the feature</li>
<li> Export &quot;as deployable feature&quot; and verify that the feature.xml
content has been updated (the reference to Qualified has the number from the
"Qualified" plugin)</li>
<li> Verify that the version number of the feature in the feature.xml has been updated</li>
</ul>
<h4>Export of nested features with qualifiers</h4>
<ul>
<li> create a new feature (no importance in the name and version)</li>
<li> add the newly created feature to this feature</li>
<li> export the "QualifiedFeature" feature and verify that the feature.xml has been updated properly</li>
</ul>
<h4>Test of optional matching rules</h4>
<ul>
<li>Create a plugin named Match1</li>
<li>Create a plugin named Match2</li>
<li>Update Match2 dependencies to optionaly requires Match1</li>
<li>Close Match1 project</li>
<li>Generate the build.xml for Match2 (right click on plugin.xml)</li>
<li>If the build.xml is generated it is ok</li>
</ul>
<h4>Tests of matching rules</h4>
<ul>
<li>Re-open the plugin named Match1</li>
<li>Update Match2 dependencies to have a dependency on Match1 which is version="1.0.0" match="perfect"</li>
<li>Generate the build.xml (right click on plugin.xml)</li>
<li>If the build.xml is generated it is ok</li>
<li>Repeat steps 2, 3 and 4 for the 3 different values for matching rules
<pre>
version="1.0.0" match="perfect"
version="1.0.0" match="equivalent"
version="1.0.0" match="compatible"
version="1.0.0" match="greaterOrEquals"
</pre>
<li>Change Match1 version number to be 1.0.1 and repeat the "repeat section". The test with perfect match is expected to
fail.</li>
<li>Change Match1 version number to be 1.1.0 and repeat the "repeat section". The tests with perfect and equivalent are
expected to fail.</li>
<li>Change Match1 version number to be 2.0.0 and repeat the "repeat section". The tests with perfect, equivalent and
compatible are expected to fail.</li>
</ul>
<h4>Export of binary plugins</h4>
<ul>
<li>Create a new feature BinFeature.</li>
<li>In this feature add the plugins Dot1, org.eclipse.swt, org.eclipse.core.runtime.</li>
<li>Export the feature and verify that the all the files from org.eclipse.swt and org.eclipse.core.runtime are here.</li>
</ul>
<h4>2.1.x test</h4>
<ul>
<li>Change your target platform to point to a 2.1.x eclipse install</li>
<li>Create a new plugin using the wizard (check that the 2.1 flag is checked by default)</li>
<li>Export the plugin and verify that no errors occured</li>
<li>Manually create the build.xml, and verify that the classpath for the compilation target starts by and boot, runtime</li>
</ul>
<h4>Test with not matched fragments</h4>
<ul>
<li>Install the nl fragments</li>
<li>Export a plugin that you have previously built and verify that no errors occured and that nothing in logged</li>
</ul>
<h4>Releng scenarios</h4>
<ul>
<li>Try to do an sdk build for win32</li>
<li>Verify that the resulting zip is of the expected size (note that there might be a difference due to doc)</li>
</ul>
<a href="#top">Back to top</a><br><hr>
<h3><a name="performance"/>Performance</h3>
<h4>Startup</h4>
<ul>
<li>follow the empty, medium and big workspace scenarios as detailed on <a href="http://eclipse.org/eclipse/development/performance/index.html">Eclipse
performance page</a>.</li>
</ul>
<a href="#top">Back to top</a><br><hr>
</body>
</html>