XP/xp/guidances/concepts/test-ideas_list.xmi - epf/org.eclipse.epf.libraries - Git at Google

 <?xml version="1.0" encoding="UTF-8"?>
 <org.eclipse.epf.uma:ContentDescription xmi:version="2.0" xmlns:xmi="http://www.omg.org/XMI" xmlns:org.eclipse.epf.uma="http://www.eclipse.org/epf/uma/1.0.3/uma.ecore" epf:version="1.0.0" xmi:id="-3i1jvKMUGGmAYPw4dHFbEg" name="test-ideas_list,8.834380241450745E-306" guid="-3i1jvKMUGGmAYPw4dHFbEg" changeDate="2006-12-01T15:44:08.749-0800" version="1.0.0">
   <mainDescription>&lt;h3&gt;
     &lt;a id=&quot;Introduction&quot; name=&quot;Introduction&quot;&gt;Introduction&lt;/a&gt;
 &lt;/h3&gt;
 &lt;p&gt;
     Information used in designing tests is gathered from many places: design models, classifier interfaces, statecharts,
     and code itself. At some point, this source document information must be transformed into executable tests:
 &lt;/p&gt;
 &lt;ul&gt;
     &lt;li&gt;
         specific inputs given to the software under test
     &lt;/li&gt;
     &lt;li&gt;
         in a particular hardware and software configuration
     &lt;/li&gt;
     &lt;li&gt;
         initialized to a known state
     &lt;/li&gt;
     &lt;li&gt;
         with specific results expected
     &lt;/li&gt;
 &lt;/ul&gt;
 &lt;p&gt;
     It's possible to go directly from source document information to executable tests, but it's often useful to add an
     intermediate step. In this step, test ideas are written into a &lt;i&gt;Test-Ideas List&lt;/i&gt;, which is used to create
     executable tests.
 &lt;/p&gt;
 &lt;h3&gt;
     &lt;a id=&quot;TestIdeas&quot; name=&quot;TestIdeas&quot;&gt;What are Test Ideas?&lt;/a&gt;
 &lt;/h3&gt;
 &lt;p&gt;
     A test idea (sometimes referred to as a test requirement) is a brief statement about a test that could be performed. As
     a simple example, let's consider a function that calculates a square root and come up with some test ideas:
 &lt;/p&gt;
 &lt;ul&gt;
     &lt;li&gt;
         give a number that's barely less than zero as input
     &lt;/li&gt;
     &lt;li&gt;
         give zero as the input
     &lt;/li&gt;
     &lt;li&gt;
         test a number that's a perfect square, like 4 or 16 (is the result exactly 2 or 4?)
     &lt;/li&gt;
 &lt;/ul&gt;
 &lt;p&gt;
     Each of these ideas could readily be converted into an executable test with exact descriptions of inputs and expected
     results.
 &lt;/p&gt;
 &lt;p&gt;
     There are two advantages to this less-specific intermediate form:
 &lt;/p&gt;
 &lt;ul&gt;
     &lt;li&gt;
         test ideas are more reviewable and understandable than complete tests - it's easier to understand the reasoning
         behind them
     &lt;/li&gt;
     &lt;li&gt;
         test ideas support more powerful tests, as described later under the heading &lt;a href=&quot;#TestDesignUsingTheList&quot;&gt;Test
         Design Using the List&lt;/a&gt;
     &lt;/li&gt;
 &lt;/ul&gt;
 &lt;p&gt;
     The square root examples all describe inputs, but test ideas can describe any of the elements of an executable test.
     For example, &quot;print to a LaserJet IIIp&quot; describes an aspect of the test environment to be used for a test, as does
     &quot;test with database full&quot;, however, these latter test ideas are very incomplete in themselves: Print &lt;b&gt;what&lt;/b&gt; to the
     printer? Do &lt;b&gt;what&lt;/b&gt; with that full database? They do, however, ensure that important ideas aren't forgotten; ideas
     that will be described in more detail later in test design.
 &lt;/p&gt;
 &lt;p&gt;
     Test ideas are often based on fault&amp;nbsp;models; notions of which faults are plausible in software and how those faults
     can best be uncovered. For example, consider boundaries. It's safe to assume the square root function can be
     implemented something like this:
 &lt;/p&gt;
 &lt;blockquote&gt;
 &lt;pre&gt;
  double sqrt(double x) {     if (x &amp;lt; 0)        // signal error     ...
 &lt;/pre&gt;
 &lt;/blockquote&gt;
 &lt;p&gt;
     It's also plausible that the &lt;font size=&quot;+0&quot;&gt;&amp;lt;&lt;/font&gt; will be incorrectly typed as &lt;font size=&quot;+0&quot;&gt;&amp;lt;=&lt;/font&gt;.
     People often make that kind of mistake, so it's worth checking. The fault cannot be detected with &lt;font
     size=&quot;+0&quot;&gt;X&lt;/font&gt; having the value &lt;font size=&quot;+0&quot;&gt;2&lt;/font&gt;, because both the incorrect expression (&lt;font
     size=&quot;+0&quot;&gt;x&amp;lt;=0&lt;/font&gt;) and the correct expression (&lt;font size=&quot;+0&quot;&gt;x&amp;lt;0&lt;/font&gt;) will take the same branch of the
     &lt;font size=&quot;+0&quot;&gt;if&lt;/font&gt; statement. Similarly, giving &lt;font size=&quot;+0&quot;&gt;X&lt;/font&gt; the value -&lt;font size=&quot;+0&quot;&gt;5&lt;/font&gt;
     cannot find the fault. The only way to find it is to give &lt;font size=&quot;+0&quot;&gt;X&lt;/font&gt; the value &lt;font size=&quot;+0&quot;&gt;0&lt;/font&gt;,
     which justifies the second test idea.
 &lt;/p&gt;
 &lt;p&gt;
     In this case, the fault model is explicit. In other cases, it's implicit. For example, whenever a program manipulates a
     linked structure, it's good to test it against a circular one. It's possible that many faults could lead to a
     mishandled circular structure. For the purposes of testing, they needn't be enumerated - it suffices to know that some
     fault is likely enough that the test is worth running.
 &lt;/p&gt;
 &lt;p&gt;
     The following links provide information about getting test ideas from different kinds of fault models. The first two
     are explicit fault models; the last uses implicit ones.
 &lt;/p&gt;
 &lt;ul&gt;
     &lt;li&gt;
         &lt;a class=&quot;elementLinkWithType&quot;
         href=&quot;./../../../xp/guidances/guidelines/test_ideas_for_booleans_and_boundaries,1.7150344523489172E-305.html&quot;
         guid=&quot;1.7150344523489172E-305&quot;&gt;Guideline: Test Ideas for Booleans and Boundaries&lt;/a&gt;
     &lt;/li&gt;
     &lt;li&gt;
         &lt;a class=&quot;elementLinkWithType&quot;
         href=&quot;./../../../xp/guidances/guidelines/test_ideas_for_method_calls,8.5657170364036E-306.html&quot;
         guid=&quot;8.5657170364036E-306&quot;&gt;Guideline: Test Ideas for Method Calls&lt;/a&gt;
     &lt;/li&gt;
     &lt;li&gt;
         &lt;a class=&quot;elementLinkWithType&quot;
         href=&quot;./../../../xp/guidances/concepts/test-ideas_catalog,1.2384224477983028E-305.html&quot;
         guid=&quot;1.2384224477983028E-305&quot;&gt;Concept: Test-Ideas Catalog&lt;/a&gt;
     &lt;/li&gt;
 &lt;/ul&gt;
 &lt;p&gt;
     These fault models can be applied to many different artifacts. For example, the first one describes what to do with
     Boolean expressions. Such expressions can be found in code, in guard conditions, in statecharts and sequence diagrams,
     and in natural-language descriptions of method behaviors (such as you might find in a published API).
 &lt;/p&gt;
 &lt;p&gt;
     Occasionally it's also helpful to have guidelines for specific artifacts. See &lt;a class=&quot;elementLinkWithType&quot;
     href=&quot;./../../../xp/guidances/guidelines/test_ideas_for_statechart_and_flow_diagrams,1.0347051690476123E-305.html&quot;
     guid=&quot;1.0347051690476123E-305&quot;&gt;Guideline: Test Ideas for Statechart and Flow Diagrams&lt;/a&gt;.
 &lt;/p&gt;
 &lt;p&gt;
     A particular Test-Ideas List might contain test ideas from many fault models, and those fault models could be derived
     from more than one artifact.
 &lt;/p&gt;
 &lt;h3&gt;
     &lt;a id=&quot;TestDesignUsingTheList&quot; name=&quot;TestDesignUsingTheList&quot;&gt;Test Design Using the List&lt;/a&gt;
 &lt;/h3&gt;
 &lt;p&gt;
     Let's suppose you're designing tests for a method that searches for a string in a sequential collection. It can either
     obey case or ignore case in its search, and it returns the index of the first match found or -1 if no match is found.
 &lt;/p&gt;
 &lt;blockquote&gt;
 &lt;pre&gt;
  int Collection.find(String string,     Boolean ignoreCase);
 &lt;/pre&gt;
 &lt;/blockquote&gt;
 &lt;p&gt;
     Here are some test ideas for this method:
 &lt;/p&gt;
 &lt;ol&gt;
     &lt;li&gt;
         match found in the first position
     &lt;/li&gt;
     &lt;li&gt;
         match found in the last position
     &lt;/li&gt;
     &lt;li&gt;
         no match found
     &lt;/li&gt;
     &lt;li&gt;
         two or more matches found in the collection
     &lt;/li&gt;
     &lt;li&gt;
         case is ignored; match found, but it wouldn't match if case was obeyed
     &lt;/li&gt;
     &lt;li&gt;
         case is obeyed; an exact match is found
     &lt;/li&gt;
     &lt;li&gt;
         case is obeyed; a string that would have matched if case were ignored is skipped
     &lt;/li&gt;
 &lt;/ol&gt;
 &lt;p&gt;
     It would be simple to implement these seven tests, one for each test idea. However, different test ideas can be
     combined into a single test. For example, the following test &lt;i&gt;satisfies&lt;/i&gt; test ideas 2, 6, and 7:
 &lt;/p&gt;
 &lt;blockquote&gt;
     &lt;p&gt;
         Setup: collection initialized to [&quot;dawn&quot;, &quot;Dawn&quot;]&lt;br /&gt;
          Invocation: collection.find(&quot;Dawn&quot;, false)&lt;br /&gt;
          Expected result: return value is 1 (it would be 0 if &quot;dawn&quot; were not skipped)
     &lt;/p&gt;
 &lt;/blockquote&gt;
 &lt;p&gt;
     Making test ideas nonspecific makes them easier to combine.
 &lt;/p&gt;
 &lt;p&gt;
     It's possible to satisfy all of the test ideas in three tests. Why would three tests that satisfy seven test ideas be
     better than seven separate tests?
 &lt;/p&gt;
 &lt;ul&gt;
     &lt;li&gt;
         When you're creating a large number of simple tests, it's common to create test N+1 by copying test N and tweaking
         it just enough to satisfy the new test idea. The result, especially in more complex software, is that test N+1
         probably exercises the program in almost the same way as test N. It takes almost exactly the same path through the
         code.&lt;br /&gt;
         &lt;br /&gt;
          A smaller number of tests, each satisfying several test ideas, doesn't allow a &quot;copy and tweak&quot; approach. Each
         test will be somewhat different from the last, exercising the code in different ways and taking different
         paths.&lt;br /&gt;
         &lt;br /&gt;
          Why would that be better? If the Test-Ideas List were complete, with a test idea for every fault in the program,
         it wouldn't matter how you wrote the tests. But the list is always missing some test ideas that could find bugs. By
         having each test do very different things from the last one - by adding seemingly unneeded variety - you increase
         the chance that one of the tests will stumble over a bug by sheer dumb luck. In effect, smaller, more complex tests
         increase the chance the test will satisfy a test idea that you didn't know you needed.&lt;br /&gt;
     &lt;/li&gt;
     &lt;li&gt;
         Sometimes when you're creating more complex tests, new test ideas come to mind. That happens less often with simple
         tests, because so much of what you're doing is exactly like the last test, which dulls your mind.
     &lt;/li&gt;
 &lt;/ul&gt;
 &lt;p&gt;
     However, there are reasons for not creating complex tests.
 &lt;/p&gt;
 &lt;ul&gt;
     &lt;li&gt;
         If each test satisfies a single test idea and the test for idea 2 fails, you immediately know the most likely
         cause: the program doesn't handle a match in the last position. If a test satisfies ideas 2, 6, and 7, then
         isolating the failure is harder.&lt;br /&gt;
     &lt;/li&gt;
     &lt;li&gt;
         Complex tests are more difficult to understand and maintain. The intent of the test is less obvious.&lt;br /&gt;
     &lt;/li&gt;
     &lt;li&gt;
         Complex tests are more difficult to create. Constructing a test that satisfies five test ideas often takes more
         time than constructing five tests that each satisfy one. Moreover, it's easier to make mistakes - to think you're
         satisfying all five when you're only satisfying four.
     &lt;/li&gt;
 &lt;/ul&gt;
 &lt;p&gt;
     In practice, you must find a reasonable balance between complexity and simplicity. For example, the first tests you
     subject the software to (typically the smoke tests) should be simple, easy to understand and maintain, and intended to
     catch the most obvious problems. Later tests should be more complex, but not so complex they are not maintainable.
 &lt;/p&gt;
 &lt;p&gt;
     After you've finished a set of tests, it's good to check them against the characteristic test design mistakes discussed
     in &lt;a class=&quot;elementLinkWithType&quot;
     href=&quot;./../../../xp/guidances/concepts/developer_testing,4.085829182735815E-305.html#TestDesignMistakes&quot;
     guid=&quot;4.085829182735815E-305&quot;&gt;Concept: Developer Testing&lt;/a&gt;.
 &lt;/p&gt;
 &lt;h3&gt;
     &lt;a id=&quot;UsingTestIdeasBeforeTest&quot; name=&quot;UsingTestIdeasBeforeTest&quot;&gt;Using Test Ideas Before Testing&lt;/a&gt;
 &lt;/h3&gt;
 &lt;p&gt;
     A Test-Ideas List is useful for reviews and inspections of design artifacts. For example, consider this part of a
     design model showing the association between Department and Employee classes.
 &lt;/p&gt;
 &lt;p align=&quot;center&quot;&gt;
     &lt;img height=&quot;45&quot; alt=&quot;&quot; src=&quot;resources/tstidslst-img1.gif&quot; width=&quot;223&quot; /&gt;
 &lt;/p&gt;
 &lt;p class=&quot;picturetext&quot;&gt;
     Figure 1: Association between Department and Employee Classes
 &lt;/p&gt;
 &lt;p&gt;
     The rules for creating test ideas from such a model would ask you to consider the case where a department has many
     employees. By walking through a design and asking &quot;what if, at this point, the department has many employees?&quot;, you
     might discover design or analysis errors. For example, you might realize that only one employee at a time can be
     transferred between departments. That might be a problem if the corporation is prone to sweeping reorganizations where
     many employees need to be transferred.
 &lt;/p&gt;
 &lt;p&gt;
     Such faults, cases where a possibility was overlooked, are called &lt;i&gt;faults of omission&lt;/i&gt;. Just like the faults
     themselves, you have probably omitted tests that detect these faults from your testing effort. For example, see &lt;a
     class=&quot;elementLinkWithUserText&quot;
     href=&quot;./../../../xp/guidances/supportingmaterials/xp_and_agile_process_references,6.191633934532389E-306.html&quot;
     guid=&quot;6.191633934532389E-306&quot;&gt;[GLA81]&lt;/a&gt;, &amp;nbsp;&lt;a href=&quot;../../referenc.htm#OST84&quot;&gt;[OST84]&lt;/a&gt;, &lt;a
     href=&quot;../../referenc.htm#BAS87&quot;&gt;[BAS87]&lt;/a&gt;, &lt;a href=&quot;../../referenc.htm#MAR00&quot;&gt;[MAR00]&lt;/a&gt;, and other studies that
     show how often faults of omission escape into deployment.
 &lt;/p&gt;
 &lt;p&gt;
     The role of testing in design activities is discussed further in &lt;a class=&quot;elementLinkWithType&quot;
     href=&quot;./../../../xp/guidances/concepts/test-first_design,6.556259235358794E-306.html&quot;
     guid=&quot;6.556259235358794E-306&quot;&gt;Concept: Test-first Design&lt;/a&gt;.
 &lt;/p&gt;
 &lt;h3&gt;
     &lt;a id=&quot;TestIdeasTraceability&quot; name=&quot;TestIdeasTraceability&quot;&gt;Test Ideas and Traceability&lt;/a&gt;
 &lt;/h3&gt;
 &lt;p&gt;
     Traceability is a matter of tradeoffs. Is its value worth the cost of maintaining it? This question needs to be
     considered during &lt;a href=&quot;../../activity/ac_tst_dfnasstrcnds.htm&quot;&gt;Activity: Define Assessment and Traceability
     Needs&lt;/a&gt;.
 &lt;/p&gt;
 &lt;p&gt;
     When traceability is worthwhile, it's conventional to trace tests back to the artifacts that inspired them. For
     example, you might have traceability between an API and its tests. If the API changes, you know which tests to change.
     If the code (that implements the API) changes, you know which tests to run. If a test puzzles you, you can find the API
     it's intended to test.
 &lt;/p&gt;
 &lt;p&gt;
     The Test-Ideas List adds another level of traceability. You can trace from a test to the test ideas it satisfies, and
     then from the test ideas to the original artifact.
 &lt;/p&gt;
 &lt;br /&gt;
 &lt;br /&gt;</mainDescription>
 </org.eclipse.epf.uma:ContentDescription>
	<?xml version="1.0" encoding="UTF-8"?>
	<org.eclipse.epf.uma:ContentDescription xmi:version="2.0" xmlns:xmi="http://www.omg.org/XMI" xmlns:org.eclipse.epf.uma="http://www.eclipse.org/epf/uma/1.0.3/uma.ecore" epf:version="1.0.0" xmi:id="-3i1jvKMUGGmAYPw4dHFbEg" name="test-ideas_list,8.834380241450745E-306" guid="-3i1jvKMUGGmAYPw4dHFbEg" changeDate="2006-12-01T15:44:08.749-0800" version="1.0.0">
	<mainDescription><h3>
	<a id="Introduction" name="Introduction">Introduction</a>
	</h3>
	<p>
	Information used in designing tests is gathered from many places: design models, classifier interfaces, statecharts,
	and code itself. At some point, this source document information must be transformed into executable tests:
	</p>
	<ul>
	<li>
	specific inputs given to the software under test
	</li>
	<li>
	in a particular hardware and software configuration
	</li>
	<li>
	initialized to a known state
	</li>
	<li>
	with specific results expected
	</li>
	</ul>
	<p>
	It's possible to go directly from source document information to executable tests, but it's often useful to add an
	intermediate step. In this step, test ideas are written into a <i>Test-Ideas List</i>, which is used to create
	executable tests.
	</p>
	<h3>
	<a id="TestIdeas" name="TestIdeas">What are Test Ideas?</a>
	</h3>
	<p>
	A test idea (sometimes referred to as a test requirement) is a brief statement about a test that could be performed. As
	a simple example, let's consider a function that calculates a square root and come up with some test ideas:
	</p>
	<ul>
	<li>
	give a number that's barely less than zero as input
	</li>
	<li>
	give zero as the input
	</li>
	<li>
	test a number that's a perfect square, like 4 or 16 (is the result exactly 2 or 4?)
	</li>
	</ul>
	<p>
	Each of these ideas could readily be converted into an executable test with exact descriptions of inputs and expected
	results.
	</p>
	<p>
	There are two advantages to this less-specific intermediate form:
	</p>
	<ul>
	<li>
	test ideas are more reviewable and understandable than complete tests - it's easier to understand the reasoning
	behind them
	</li>
	<li>
	test ideas support more powerful tests, as described later under the heading <a href="#TestDesignUsingTheList">Test
	Design Using the List</a>
	</li>
	</ul>
	<p>
	The square root examples all describe inputs, but test ideas can describe any of the elements of an executable test.
	For example, "print to a LaserJet IIIp" describes an aspect of the test environment to be used for a test, as does
	"test with database full", however, these latter test ideas are very incomplete in themselves: Print <b>what</b> to the
	printer? Do <b>what</b> with that full database? They do, however, ensure that important ideas aren't forgotten; ideas
	that will be described in more detail later in test design.
	</p>
	<p>
	Test ideas are often based on fault&nbsp;models; notions of which faults are plausible in software and how those faults
	can best be uncovered. For example, consider boundaries. It's safe to assume the square root function can be
	implemented something like this:
	</p>
	<blockquote>
	<pre>
	double sqrt(double x) { if (x &lt; 0) // signal error ...
	</pre>
	</blockquote>
	<p>
	It's also plausible that the <font size="+0">&lt;</font> will be incorrectly typed as <font size="+0">&lt;=</font>.
	People often make that kind of mistake, so it's worth checking. The fault cannot be detected with <font
	size="+0">X</font> having the value <font size="+0">2</font>, because both the incorrect expression (<font
	size="+0">x&lt;=0</font>) and the correct expression (<font size="+0">x&lt;0</font>) will take the same branch of the
	<font size="+0">if</font> statement. Similarly, giving <font size="+0">X</font> the value -<font size="+0">5</font>
	cannot find the fault. The only way to find it is to give <font size="+0">X</font> the value <font size="+0">0</font>,
	which justifies the second test idea.
	</p>
	<p>
	In this case, the fault model is explicit. In other cases, it's implicit. For example, whenever a program manipulates a
	linked structure, it's good to test it against a circular one. It's possible that many faults could lead to a
	mishandled circular structure. For the purposes of testing, they needn't be enumerated - it suffices to know that some
	fault is likely enough that the test is worth running.
	</p>
	<p>
	The following links provide information about getting test ideas from different kinds of fault models. The first two
	are explicit fault models; the last uses implicit ones.
	</p>
	<ul>
	<li>
	<a class="elementLinkWithType"
	href="./../../../xp/guidances/guidelines/test_ideas_for_booleans_and_boundaries,1.7150344523489172E-305.html"
	guid="1.7150344523489172E-305">Guideline: Test Ideas for Booleans and Boundaries</a>
	</li>
	<li>
	<a class="elementLinkWithType"
	href="./../../../xp/guidances/guidelines/test_ideas_for_method_calls,8.5657170364036E-306.html"
	guid="8.5657170364036E-306">Guideline: Test Ideas for Method Calls</a>
	</li>
	<li>
	<a class="elementLinkWithType"
	href="./../../../xp/guidances/concepts/test-ideas_catalog,1.2384224477983028E-305.html"
	guid="1.2384224477983028E-305">Concept: Test-Ideas Catalog</a>
	</li>
	</ul>
	<p>
	These fault models can be applied to many different artifacts. For example, the first one describes what to do with
	Boolean expressions. Such expressions can be found in code, in guard conditions, in statecharts and sequence diagrams,
	and in natural-language descriptions of method behaviors (such as you might find in a published API).
	</p>
	<p>
	Occasionally it's also helpful to have guidelines for specific artifacts. See <a class="elementLinkWithType"
	href="./../../../xp/guidances/guidelines/test_ideas_for_statechart_and_flow_diagrams,1.0347051690476123E-305.html"
	guid="1.0347051690476123E-305">Guideline: Test Ideas for Statechart and Flow Diagrams</a>.
	</p>
	<p>
	A particular Test-Ideas List might contain test ideas from many fault models, and those fault models could be derived
	from more than one artifact.
	</p>
	<h3>
	<a id="TestDesignUsingTheList" name="TestDesignUsingTheList">Test Design Using the List</a>
	</h3>
	<p>
	Let's suppose you're designing tests for a method that searches for a string in a sequential collection. It can either
	obey case or ignore case in its search, and it returns the index of the first match found or -1 if no match is found.
	</p>
	<blockquote>
	<pre>
	int Collection.find(String string, Boolean ignoreCase);
	</pre>
	</blockquote>
	<p>
	Here are some test ideas for this method:
	</p>
	<ol>
	<li>
	match found in the first position
	</li>
	<li>
	match found in the last position
	</li>
	<li>
	no match found
	</li>
	<li>
	two or more matches found in the collection
	</li>
	<li>
	case is ignored; match found, but it wouldn't match if case was obeyed
	</li>
	<li>
	case is obeyed; an exact match is found
	</li>
	<li>
	case is obeyed; a string that would have matched if case were ignored is skipped
	</li>
	</ol>
	<p>
	It would be simple to implement these seven tests, one for each test idea. However, different test ideas can be
	combined into a single test. For example, the following test <i>satisfies</i> test ideas 2, 6, and 7:
	</p>
	<blockquote>
	<p>
	Setup: collection initialized to ["dawn", "Dawn"]<br />
	Invocation: collection.find("Dawn", false)<br />
	Expected result: return value is 1 (it would be 0 if "dawn" were not skipped)
	</p>
	</blockquote>
	<p>
	Making test ideas nonspecific makes them easier to combine.
	</p>
	<p>
	It's possible to satisfy all of the test ideas in three tests. Why would three tests that satisfy seven test ideas be
	better than seven separate tests?
	</p>
	<ul>
	<li>
	When you're creating a large number of simple tests, it's common to create test N+1 by copying test N and tweaking
	it just enough to satisfy the new test idea. The result, especially in more complex software, is that test N+1
	probably exercises the program in almost the same way as test N. It takes almost exactly the same path through the
	code.<br />
	<br />
	A smaller number of tests, each satisfying several test ideas, doesn't allow a "copy and tweak" approach. Each
	test will be somewhat different from the last, exercising the code in different ways and taking different
	paths.<br />
	<br />
	Why would that be better? If the Test-Ideas List were complete, with a test idea for every fault in the program,
	it wouldn't matter how you wrote the tests. But the list is always missing some test ideas that could find bugs. By
	having each test do very different things from the last one - by adding seemingly unneeded variety - you increase
	the chance that one of the tests will stumble over a bug by sheer dumb luck. In effect, smaller, more complex tests
	increase the chance the test will satisfy a test idea that you didn't know you needed.<br />
	</li>
	<li>
	Sometimes when you're creating more complex tests, new test ideas come to mind. That happens less often with simple
	tests, because so much of what you're doing is exactly like the last test, which dulls your mind.
	</li>
	</ul>
	<p>
	However, there are reasons for not creating complex tests.
	</p>
	<ul>
	<li>
	If each test satisfies a single test idea and the test for idea 2 fails, you immediately know the most likely
	cause: the program doesn't handle a match in the last position. If a test satisfies ideas 2, 6, and 7, then
	isolating the failure is harder.<br />
	</li>
	<li>
	Complex tests are more difficult to understand and maintain. The intent of the test is less obvious.<br />
	</li>
	<li>
	Complex tests are more difficult to create. Constructing a test that satisfies five test ideas often takes more
	time than constructing five tests that each satisfy one. Moreover, it's easier to make mistakes - to think you're
	satisfying all five when you're only satisfying four.
	</li>
	</ul>
	<p>
	In practice, you must find a reasonable balance between complexity and simplicity. For example, the first tests you
	subject the software to (typically the smoke tests) should be simple, easy to understand and maintain, and intended to
	catch the most obvious problems. Later tests should be more complex, but not so complex they are not maintainable.
	</p>
	<p>
	After you've finished a set of tests, it's good to check them against the characteristic test design mistakes discussed
	in <a class="elementLinkWithType"
	href="./../../../xp/guidances/concepts/developer_testing,4.085829182735815E-305.html#TestDesignMistakes"
	guid="4.085829182735815E-305">Concept: Developer Testing</a>.
	</p>
	<h3>
	<a id="UsingTestIdeasBeforeTest" name="UsingTestIdeasBeforeTest">Using Test Ideas Before Testing</a>
	</h3>
	<p>
	A Test-Ideas List is useful for reviews and inspections of design artifacts. For example, consider this part of a
	design model showing the association between Department and Employee classes.
	</p>
	<p align="center">
	<img height="45" alt="" src="resources/tstidslst-img1.gif" width="223" />
	</p>
	<p class="picturetext">
	Figure 1: Association between Department and Employee Classes
	</p>
	<p>
	The rules for creating test ideas from such a model would ask you to consider the case where a department has many
	employees. By walking through a design and asking "what if, at this point, the department has many employees?", you
	might discover design or analysis errors. For example, you might realize that only one employee at a time can be
	transferred between departments. That might be a problem if the corporation is prone to sweeping reorganizations where
	many employees need to be transferred.
	</p>
	<p>
	Such faults, cases where a possibility was overlooked, are called <i>faults of omission</i>. Just like the faults
	themselves, you have probably omitted tests that detect these faults from your testing effort. For example, see <a
	class="elementLinkWithUserText"
	href="./../../../xp/guidances/supportingmaterials/xp_and_agile_process_references,6.191633934532389E-306.html"
	guid="6.191633934532389E-306">[GLA81]</a>, &nbsp;<a href="../../referenc.htm#OST84">[OST84]</a>, <a
	href="../../referenc.htm#BAS87">[BAS87]</a>, <a href="../../referenc.htm#MAR00">[MAR00]</a>, and other studies that
	show how often faults of omission escape into deployment.
	</p>
	<p>
	The role of testing in design activities is discussed further in <a class="elementLinkWithType"
	href="./../../../xp/guidances/concepts/test-first_design,6.556259235358794E-306.html"
	guid="6.556259235358794E-306">Concept: Test-first Design</a>.
	</p>
	<h3>
	<a id="TestIdeasTraceability" name="TestIdeasTraceability">Test Ideas and Traceability</a>
	</h3>
	<p>
	Traceability is a matter of tradeoffs. Is its value worth the cost of maintaining it? This question needs to be
	considered during <a href="../../activity/ac_tst_dfnasstrcnds.htm">Activity: Define Assessment and Traceability
	Needs</a>.
	</p>
	<p>
	When traceability is worthwhile, it's conventional to trace tests back to the artifacts that inspired them. For
	example, you might have traceability between an API and its tests. If the API changes, you know which tests to change.
	If the code (that implements the API) changes, you know which tests to run. If a test puzzles you, you can find the API
	it's intended to test.
	</p>
	<p>
	The Test-Ideas List adds another level of traceability. You can trace from a test to the test ideas it satisfies, and
	then from the test ideas to the original artifact.
	</p>
	<br />
	<br /></mainDescription>
	</org.eclipse.epf.uma:ContentDescription>