wst/components/xml/M4/milestone_test_plan.xml - gerrit/www.eclipse.org/webtools - Git at Google

 <?xml version="1.0" encoding="utf-8"?>
 <?xml-stylesheet type="text/xsl" href="../../../../wtp.xsl"?>
 <html>
 	<head>
 		<meta name="root" content="../../../../../" />
 		<title>WTP Milestone 4</title>
 	</head>
 	<body>
 		<h1>XML Test Plan</h1>
 		<h2>Status of this Plan</h2>
 		<p>Proposed Plan (4-14-05)</p>

 		<h2>Overall goals</h2>
 		<h3><b>New for M4</b></h3>
 		<p>
 			Not much has been added in terms of new functionality
 			for M4.  Most of the work done was in the areas of
 			refactoring, cleaning up dead code, and cleaning up APIs.
 			Also more unit tests were added which are run regularly
 			with the build.
 			<br/><br/>
 			The M3 test plan should have sufficient coverage for M4.
 		</p>
 		<h3><b>Co-developer Testing</b></h3>
 		<p>
 			We will inspect &quot;runtime&quot; version of build to be
 			sure extra source is not included, and more important, we'll
 			inspect and test importing SDK version to be sure all
 			relevant &quot;open source&quot; is included in that SDK
 			build and that it correctly imports into a development
 			environment.
 		</p>
 		<h3><b>API Testing</b></h3>
 		<p>
 			Here in M4 we don't consider we have any official API yet
 			(since not spec'd as such) but will reserve this space for
 			future plans to details were API Unit tests are, their
 			coverage, etc.
 		</p>
 		<p>
 			We do have several hundred unit tests which we expect to be
 			running and passing for M4, which test various
 			aspects of parsing, model creation, and correct charset
 			handling, among other things.
 		</p>

 		<h3><b>End User Testing</b></h3>
 		<p>
 			The nature of the end-user
 			testing is intentionally planned to be "ad hoc" instead of
 			specifying step by step "how to" directions and specific
 			"expected results" sections often seen in test cases. This
 			is done because its felt leads to greater number of "paths"
 			being tested, and allows end-users more motivation for
 			logging "bugs" if things didn't work as
 			<i>they</i> expected, even if it is working as designed.
 		</p>

 		<p>
 			As we progress through milestones, we'll add more and more
 			detail for special cases, special files, special projects,
 			etc.When we do have special or sample test files and
 			projects, we will keep those stored in CVS, as projects
 			under a
 			<i>testdata</i>
 			directory under the
 			<i>development</i>
 			directory of relevant component so that testers (from
 			immediate team, or community) can easily check out into the
 			environment being tested.
 		</p>

 		<h3><b>Platform Testing</b></h3>
 		<p>
 			While we do not have any platform specific code, or
 			function, we will have some team members do end-user tests
 			on Linux, some on Windows. We will also confirm unit tests
 			pass on both platforms.
 		</p>

 		<h3><b>Performance Testing</b></h3>
 		<p>
 			We have added (some) automated performance tests along the lines
 			of the Eclipse base performance unit tests in future
 			milestones.  These are currently in the <b>org.eclipse.wst.*.ui.tests.performance</b>
 			and <b>org.eclipse.jst.jsp.ui.tests.performance</b> plugins.
 			<br/>
 			<br/>
 			We will continue to add more test cases in upcoming milestones.
 		</p>

 		<h2>XML Tests</h2>
 		<ul>
 			<li>
 				check Source Editing features from feature <a href="../../../../wst/components/sse/M4/milestone_test_plan.xml#matrix">matrix</a>
 				<br />
 				check samples of range of types of web resources
 				(web.xml, tld files, schema source, etc)
 				<br />
 				especially any we (WTP) generate
 			</li>
 			<li>
 				test Design page, Source page, Outline view, Properties
 				view synchronization
 			</li>
 			<li>
 				check content assist and source validation using content
 				model from the following source:
 				<ul>
 					<li>
 						inferred content model (XML with no DOCTYPE or
 						schema specified)
 					</li>
 					<li>DTD from XML catalog</li>
 					<li>DTD from URI</li>
 					<li>schema from XML catalog</li>
 					<li>schema from URI</li>
 					<li>
 						basic schema (with lots of complex types, and
 						lots of 'includes')
 					</li>
 					<li>advanced schema (with lots of namespaces)</li>
 				</ul>
 			</li>
 			<li>
 				A quick sanity check that preferences have effect, and
 				do no harm. Only quick check, since many changes planned
 				in this are for future milestones.
 			</li>
 		</ul>

 		<h2>Specific Test Items for M4</h2>

 		<p>

 			<ul>
 				<li>
 					as you type validation:
 					<ul>
 						<li>
 							job based, make sure squiggles show up when
 							they should and are removed when problems
 							are fixed
 						</li>
 						<li>
 							test various partition types in the document
 						</li>
 						<li>new Xerces-style validators (for as-you-type)</li>
 						<li>XSD validator (for as-you-type)</li>
 					</ul>
 				</li>

 				<li>
 					hyperlink open on:
 					<ul>
 						<li>schema location, dtd location</li>
 					</ul>
 				</li>

 				<li>
 					preferences:
 					<ul>
 						<li>how we use the preferences in the "All Text Editors"
 					         preference page now.
 					    </li>
 						<li>make sure preferenes work</li>
 						<li>after shutdown and restart</li>
 					</ul>
 				</li>

 				<li>
 					run performance unit tests:
 					<ul>
 						<li>
 							record results for comparison with future
 							milestones
 						</li>
 					</ul>
 				</li>

 				<li>
 					profiling:
 					<ul>
 						<li>test basic editor functions and look for
 							problem areas (large memory consumption, intense cpu usage)
 						</li>
 					</ul>
 				</li>

 				<li>
 					sleak:
 					<ul>
 						<li>
 							run through basic editor functions and look
 							for memory leaks
 						</li>
 					</ul>
 				</li>w
 			</ul>
 		</p>

 		<h2><b>Regression Tests</b></h2>
 		<p>
 			<a href="../../../../wst/components/sse/M4/formatting-test.html">Formatting tests</a>
 			<br/>
 		</p>

 		<h2>Source Editing Test Plans</h2>
 		<p>
 			<a href="../../../../wst/components/sse/M4/milestone_test_plan.html">org.eclipse.wst.sse</a>
 			<br/><a href="../../../../wst/components/xml/M4/milestone_test_plan.html">org.eclipse.wst.xml</a>
 			<br/><a href="../../../../wst/components/html/M4/milestone_test_plan.html">org.eclipse.wst.html</a>
 			<br/><a href="../../../../wst/components/css/M4/milestone_test_plan.html">org.eclipse.wst.css</a>
 			<br/><a href="../../../../wst/components/dtd/M4/milestone_test_plan.html">org.eclipse.wst.dtd</a>
 			<br/><a href="../../../../wst/components/javascript/M4/milestone_test_plan.html">org.eclipse.wst.javascript</a>

 			<br/><a href="../../../../jst/components/jsp/M4/milestone_test_plan.html">org.eclipse.jst.jsp</a>
 		</p>
 	</body>
 </html>
	<?xml version="1.0" encoding="utf-8"?>
	<?xml-stylesheet type="text/xsl" href="../../../../wtp.xsl"?>
	<html>
	<head>
	<meta name="root" content="../../../../../" />
	<title>WTP Milestone 4</title>
	</head>
	<body>
	<h1>XML Test Plan</h1>
	<h2>Status of this Plan</h2>
	<p>Proposed Plan (4-14-05)</p>

	<h2>Overall goals</h2>
	<h3><b>New for M4</b></h3>
	<p>
	Not much has been added in terms of new functionality
	for M4. Most of the work done was in the areas of
	refactoring, cleaning up dead code, and cleaning up APIs.
	Also more unit tests were added which are run regularly
	with the build.
	<br/><br/>
	The M3 test plan should have sufficient coverage for M4.
	</p>
	<h3><b>Co-developer Testing</b></h3>
	<p>
	We will inspect "runtime" version of build to be
	sure extra source is not included, and more important, we'll
	inspect and test importing SDK version to be sure all
	relevant "open source" is included in that SDK
	build and that it correctly imports into a development
	environment.
	</p>
	<h3><b>API Testing</b></h3>
	<p>
	Here in M4 we don't consider we have any official API yet
	(since not spec'd as such) but will reserve this space for
	future plans to details were API Unit tests are, their
	coverage, etc.
	</p>
	<p>
	We do have several hundred unit tests which we expect to be
	running and passing for M4, which test various
	aspects of parsing, model creation, and correct charset
	handling, among other things.
	</p>

	<h3><b>End User Testing</b></h3>
	<p>
	The nature of the end-user
	testing is intentionally planned to be "ad hoc" instead of
	specifying step by step "how to" directions and specific
	"expected results" sections often seen in test cases. This
	is done because its felt leads to greater number of "paths"
	being tested, and allows end-users more motivation for
	logging "bugs" if things didn't work as
	<i>they</i> expected, even if it is working as designed.
	</p>

	<p>
	As we progress through milestones, we'll add more and more
	detail for special cases, special files, special projects,
	etc.When we do have special or sample test files and
	projects, we will keep those stored in CVS, as projects
	under a
	<i>testdata</i>
	directory under the
	<i>development</i>
	directory of relevant component so that testers (from
	immediate team, or community) can easily check out into the
	environment being tested.
	</p>

	<h3><b>Platform Testing</b></h3>
	<p>
	While we do not have any platform specific code, or
	function, we will have some team members do end-user tests
	on Linux, some on Windows. We will also confirm unit tests
	pass on both platforms.
	</p>

	<h3><b>Performance Testing</b></h3>
	<p>
	We have added (some) automated performance tests along the lines
	of the Eclipse base performance unit tests in future
	milestones. These are currently in the <b>org.eclipse.wst.*.ui.tests.performance</b>
	and <b>org.eclipse.jst.jsp.ui.tests.performance</b> plugins.
	<br/>
	<br/>
	We will continue to add more test cases in upcoming milestones.
	</p>

	<h2>XML Tests</h2>
	<ul>
	<li>
	check Source Editing features from feature <a href="../../../../wst/components/sse/M4/milestone_test_plan.xml#matrix">matrix</a>
	<br />
	check samples of range of types of web resources
	(web.xml, tld files, schema source, etc)
	<br />
	especially any we (WTP) generate
	</li>
	<li>
	test Design page, Source page, Outline view, Properties
	view synchronization
	</li>
	<li>
	check content assist and source validation using content
	model from the following source:
	<ul>
	<li>
	inferred content model (XML with no DOCTYPE or
	schema specified)
	</li>
	<li>DTD from XML catalog</li>
	<li>DTD from URI</li>
	<li>schema from XML catalog</li>
	<li>schema from URI</li>
	<li>
	basic schema (with lots of complex types, and
	lots of 'includes')
	</li>
	<li>advanced schema (with lots of namespaces)</li>
	</ul>
	</li>
	<li>
	A quick sanity check that preferences have effect, and
	do no harm. Only quick check, since many changes planned
	in this are for future milestones.
	</li>
	</ul>

	<h2>Specific Test Items for M4</h2>

	<p>

	<ul>
	<li>
	as you type validation:
	<ul>
	<li>
	job based, make sure squiggles show up when
	they should and are removed when problems
	are fixed
	</li>
	<li>
	test various partition types in the document
	</li>
	<li>new Xerces-style validators (for as-you-type)</li>
	<li>XSD validator (for as-you-type)</li>
	</ul>
	</li>

	<li>
	hyperlink open on:
	<ul>
	<li>schema location, dtd location</li>
	</ul>
	</li>

	<li>
	preferences:
	<ul>
	<li>how we use the preferences in the "All Text Editors"
	preference page now.
	</li>
	<li>make sure preferenes work</li>
	<li>after shutdown and restart</li>
	</ul>
	</li>

	<li>
	run performance unit tests:
	<ul>
	<li>
	record results for comparison with future
	milestones
	</li>
	</ul>
	</li>

	<li>
	profiling:
	<ul>
	<li>test basic editor functions and look for
	problem areas (large memory consumption, intense cpu usage)
	</li>
	</ul>
	</li>

	<li>
	sleak:
	<ul>
	<li>
	run through basic editor functions and look
	for memory leaks
	</li>
	</ul>
	</li>w
	</ul>
	</p>

	<h2><b>Regression Tests</b></h2>
	<p>
	<a href="../../../../wst/components/sse/M4/formatting-test.html">Formatting tests</a>
	<br/>
	</p>

	<h2>Source Editing Test Plans</h2>
	<p>
	<a href="../../../../wst/components/sse/M4/milestone_test_plan.html">org.eclipse.wst.sse</a>
	<br/><a href="../../../../wst/components/xml/M4/milestone_test_plan.html">org.eclipse.wst.xml</a>
	<br/><a href="../../../../wst/components/html/M4/milestone_test_plan.html">org.eclipse.wst.html</a>
	<br/><a href="../../../../wst/components/css/M4/milestone_test_plan.html">org.eclipse.wst.css</a>
	<br/><a href="../../../../wst/components/dtd/M4/milestone_test_plan.html">org.eclipse.wst.dtd</a>
	<br/><a href="../../../../wst/components/javascript/M4/milestone_test_plan.html">org.eclipse.wst.javascript</a>

	<br/><a href="../../../../jst/components/jsp/M4/milestone_test_plan.html">org.eclipse.jst.jsp</a>
	</p>
	</body>
	</html>