<?xml version="1.0" encoding="UTF-8"?>
<org.eclipse.epf.uma:ContentDescription xmi:version="2.0"
    xmlns:xmi="http://www.omg.org/XMI" xmlns:org.eclipse.epf.uma="http://www.eclipse.org/epf/uma/1.0.4/uma.ecore"
    xmlns:epf="http://www.eclipse.org/epf" epf:version="1.2.0" xmi:id="-nwZMQTZtIwI5weh9c_HoYA"
    name="test_ideas_for_method_calls,8.5657170364036E-306" guid="-nwZMQTZtIwI5weh9c_HoYA"
    changeDate="2006-11-21T18:58:59.045-0500" version="1.0.0">
  <mainDescription>&lt;a id=&quot;XE_test__developer_testing__test_ideas__for_method_calls&quot; name=&quot;XE_test__developer_testing__test_ideas__for_method_calls&quot;>&lt;/a>&lt;a id=&quot;XE_design__developer_testing__test_ideas__for_method_calls&quot; name=&quot;XE_design__developer_testing__test_ideas__for_method_calls&quot;>&lt;/a>&lt;a id=&quot;XE_test-ideas__for_method_calls&quot; name=&quot;XE_test-ideas__for_method_calls&quot;>&lt;/a> &#xD;
&lt;h3>&#xD;
    &lt;a id=&quot;Introduction&quot; name=&quot;Introduction&quot;>Introduction&lt;/a>&#xD;
&lt;/h3>&#xD;
&lt;p>&#xD;
    Here's an example of defective code:&#xD;
&lt;/p>&#xD;
&lt;blockquote>&#xD;
&lt;pre>&#xD;
File file = new File(stringName);&#xD;
file.delete();&#xD;
&lt;/pre>&#xD;
&lt;/blockquote>&#xD;
&lt;p>&#xD;
    The defect is that &lt;font size=&quot;+0&quot;>File.delete&lt;/font> can fail, but the code doesn't check for that. Fixing it requires&#xD;
    the addition of the italicized code shown here:&#xD;
&lt;/p>&#xD;
&lt;blockquote>&#xD;
&lt;pre>&#xD;
File file = new File(stringName);&#xD;
&lt;font color=&quot;#ff0000&quot;>&#xD;
&lt;i>&#xD;
&lt;b>&#xD;
if (&lt;/b>&lt;/i>&lt;/font>file.delete()&lt;font color=&quot;#ff0000&quot;>&#xD;
&lt;i>&#xD;
&lt;b>&#xD;
== false) {...}&lt;/b>&lt;/i>&lt;/font>&#xD;
&lt;/pre>&#xD;
&lt;/blockquote>&#xD;
&lt;p>&#xD;
    This guideline describes a method for detecting cases where your code does not handle the result of calling a method.&#xD;
    (Note that it assumes that the method called produces the correct result for whatever inputs you give it. That's&#xD;
    something that should be tested, but creating test ideas for the called method is a separate activity. That is, it's&#xD;
    not your job to test &lt;font size=&quot;+0&quot;>File.delete&lt;/font>.)&#xD;
&lt;/p>&#xD;
&lt;p>&#xD;
    The key notion is that you should create a test idea for each &lt;i>distinct unhandled relevant result&lt;/i> of a method&#xD;
    call. To define that term, let's first look at &lt;i>result&lt;/i>. When a method executes, it changes the state of the&#xD;
    world. Here are some examples:&#xD;
&lt;/p>&#xD;
&lt;ul>&#xD;
    &lt;li>&#xD;
        It might push return values on the runtime stack.&#xD;
    &lt;/li>&#xD;
    &lt;li>&#xD;
        It might throw an exception.&#xD;
    &lt;/li>&#xD;
    &lt;li>&#xD;
        It might change a global variable.&#xD;
    &lt;/li>&#xD;
    &lt;li>&#xD;
        It might update a record in a database.&#xD;
    &lt;/li>&#xD;
    &lt;li>&#xD;
        It might send data over the network.&#xD;
    &lt;/li>&#xD;
    &lt;li>&#xD;
        It might print a message to standard output.&#xD;
    &lt;/li>&#xD;
&lt;/ul>&#xD;
&lt;p>&#xD;
    Now let's look at &lt;i>relevant&lt;/i>, again using some examples.&#xD;
&lt;/p>&#xD;
&lt;ul>&#xD;
    &lt;li>&#xD;
        Suppose the method being called prints a message to standard output. That &quot;changes the state of the world&quot;, but it&#xD;
        cannot affect the further processing of this program. No matter what gets printed, even nothing at all, it can't&#xD;
        affect the execution of your code.&#xD;
    &lt;/li>&#xD;
    &lt;li>&#xD;
        If the method returns true for success and false for failure, your program very likely should branch based on the&#xD;
        result. So that return value is relevant.&#xD;
    &lt;/li>&#xD;
    &lt;li>&#xD;
        If the called method updates a database record that your code later reads and uses, the result (updating the&#xD;
        record) is relevant.&#xD;
    &lt;/li>&#xD;
&lt;/ul>&#xD;
&lt;p>&#xD;
    (There's no absolute line between relevant and irrelevant. By calling &lt;font size=&quot;+0&quot;>print&lt;/font>, your method might&#xD;
    cause buffers to be allocated, and that allocation might be relevant after &lt;font size=&quot;+0&quot;>print&lt;/font> returns. It's&#xD;
    conceivable that a defect might depend on whether and what buffers were allocated. It's conceivable, but is it at all&#xD;
    plausible?)&#xD;
&lt;/p>&#xD;
&lt;p>&#xD;
    A method might often have a very large number of results, but only some of them will be &lt;i>distinct&lt;/i>. For example,&#xD;
    consider a method that writes bytes to disk. It might return a number less than zero to indicate failure; otherwise, it&#xD;
    returns the number of bytes written (which might be fewer than the number requested). The large number of possibilities&#xD;
    can be grouped into three distinct results:&#xD;
&lt;/p>&#xD;
&lt;ul>&#xD;
    &lt;li>&#xD;
        a number less than zero.&#xD;
    &lt;/li>&#xD;
    &lt;li>&#xD;
        the number written equals the number requested&#xD;
    &lt;/li>&#xD;
    &lt;li>&#xD;
        some bytes were written, but less than the number requested.&#xD;
    &lt;/li>&#xD;
&lt;/ul>&#xD;
&lt;p>&#xD;
    All the values less than zero are grouped into one result because no reasonable program will make a distinction among&#xD;
    them. All of them (if, indeed, more than one is possible) should be treated as an error. Similarly, if the code&#xD;
    requested that 500 bytes be written, it doesn't matter if 34 were actually written or 340: the same thing will probably&#xD;
    be done with the unwritten bytes. (If something different should be done for some value, such as 0, that will form a&#xD;
    new distinct result.)&#xD;
&lt;/p>&#xD;
&lt;p>&#xD;
    There's one last word in the defining term to explain. This particular testing technique is not concerned with distinct&#xD;
    results that are already &lt;i>handled&lt;/i>. Consider, again, this code:&#xD;
&lt;/p>&#xD;
&lt;blockquote>&#xD;
&lt;pre>&#xD;
File file = new File(stringName);&#xD;
if (file.delete() == false) {...}&#xD;
&lt;/pre>&#xD;
&lt;/blockquote>&#xD;
&lt;p>&#xD;
    There are two distinct results (true and false). The code handles them. It might handle them incorrectly, but test&#xD;
    ideas from &lt;a class=&quot;elementLinkWithType&quot; href=&quot;./../../../xp/guidances/guidelines/test_ideas_for_booleans_and_boundaries.html&quot; guid=&quot;1.7150344523489172E-305&quot;>Guideline: Test Ideas for Booleans and Boundaries&lt;/a> will check that. This test&#xD;
    technique is concerned with distinct results that are not specifically handled by distinct code. That might happen for&#xD;
    two reasons: you thought the distinction was irrelevant, or you simply overlooked it. Here's an example of the first&#xD;
    case:&#xD;
&lt;/p>&#xD;
&lt;blockquote>&#xD;
&lt;pre>&#xD;
result = m.method();&#xD;
switch (result) {&#xD;
    case FAIL:&#xD;
    case CRASH:&#xD;
       ...&#xD;
       break;&#xD;
    case DEFER:&#xD;
       ...&#xD;
       break;&#xD;
    default:&#xD;
       ...&#xD;
       break;&#xD;
}&#xD;
&lt;/pre>&#xD;
&lt;/blockquote>&#xD;
&lt;p>&#xD;
    &lt;font size=&quot;+0&quot;>FAIL&lt;/font> and &lt;font size=&quot;+0&quot;>CRASH&lt;/font> are handled by the same code. It might be wise to check&#xD;
    that that's really appropriate. Here's an example of an overlooked distinction:&#xD;
&lt;/p>&#xD;
&lt;blockquote>&#xD;
&lt;pre>&#xD;
result = s.shutdown();&#xD;
if (result == PANIC) {&#xD;
   ...&#xD;
} else {&#xD;
   // success! Shut down the reactor.&#xD;
   ...&#xD;
} &#xD;
&lt;/pre>&#xD;
&lt;/blockquote>&#xD;
&lt;p>&#xD;
    It turns out that shutdown can return an additional distinct result: &lt;font size=&quot;+0&quot;>RETRY&lt;/font>. The code as written&#xD;
    treats that case the same as the success case, which is almost certainly wrong.&#xD;
&lt;/p>&#xD;
&lt;h3>&#xD;
    &lt;a id=&quot;FindingTestIdeas&quot; name=&quot;FindingTestIdeas&quot;>Finding test ideas&lt;/a>&#xD;
&lt;/h3>&#xD;
&lt;p>&#xD;
    So your goal is to think of those distinct relevant results you previously overlooked. That seems impossible: why would&#xD;
    you realize they're relevant now if you didn't earlier?&#xD;
&lt;/p>&#xD;
&lt;p>&#xD;
    The answer is that a systematic re-examination of your code, when in a testing frame of mind and not a programming&#xD;
    frame of mind, can sometimes cause you to think new thoughts. You &lt;i>can&lt;/i> question your own assumptions by&#xD;
    methodically stepping through your code, looking at the methods you call, rechecking their documentation, and thinking.&#xD;
    Here are some cases to watch for.&#xD;
&lt;/p>&#xD;
&lt;h4>&#xD;
    &quot;Impossible&quot; cases&#xD;
&lt;/h4>&#xD;
&lt;p>&#xD;
    Often, it will appear that error returns are impossible. Doublecheck your assumptions.&#xD;
&lt;/p>&#xD;
&lt;p>&#xD;
    This example shows a Java implementation of a common Unix idiom for handling temporary files.&#xD;
&lt;/p>&#xD;
&lt;blockquote>&#xD;
&lt;pre>&#xD;
File file = new File(&quot;tempfile&quot;);&#xD;
FileOutputStream s;&#xD;
try {&#xD;
    // open the temp file.&#xD;
    s = new FileOutputStream(file);&#xD;
} catch (IOException e) {...}&#xD;
// Make sure temp file will be deleted&#xD;
file.delete();&#xD;
&lt;/pre>&#xD;
&lt;/blockquote>&#xD;
&lt;p>&#xD;
    The goal is to make sure that a temporary file is always deleted, no matter how the program exits. You do this by&#xD;
    creating the temporary file, then immediately deleting it. On Unix, you can continue to work with the deleted file, and&#xD;
    the operating system takes care of cleaning up when the process exits. A not-painstaking Unix programmer might not&#xD;
    write the code to check for a failed deletion. Since she just successfully created the file, she must be able to delete&#xD;
    it.&#xD;
&lt;/p>&#xD;
&lt;p>&#xD;
    This trick doesn't work on Windows. The deletion will fail because the file is open. Discovering that fact is hard: as&#xD;
    of August 2000, the Java documentation did not enumerate the situations in which &lt;font size=&quot;+0&quot;>delete&lt;/font> could&#xD;
    fail; it merely says that it can. But-perhaps-when in &quot;testing mode&quot;, the programmer might question her assumption.&#xD;
    Since her code is supposed to be &quot;write once, run everywhere&quot;, she might ask a Windows programmer when &lt;font     size=&quot;+0&quot;>File.delete&lt;/font> fails on Windows and so discover the awful truth.&#xD;
&lt;/p>&#xD;
&lt;h4>&#xD;
    &quot;Irrelevant&quot; cases&#xD;
&lt;/h4>&#xD;
&lt;p>&#xD;
    Another force against noticing a distinct relevant value is being already convinced that it doesn't matter. A Java&#xD;
    &lt;font size=&quot;+0&quot;>Comparator&lt;/font>'s &lt;font size=&quot;+0&quot;>compare&lt;/font> method returns either a number &amp;lt;0, 0, or a number&#xD;
    &amp;gt;0. Those are three distinct cases that might be tried. This code lumps two of them together:&#xD;
&lt;/p>&#xD;
&lt;blockquote>&#xD;
&lt;pre>&#xD;
void allCheck(Comparator c) {&#xD;
   ...&#xD;
   if (c.compare(o1, o2) &amp;lt;= 0) {&#xD;
      ...&#xD;
   } else {&#xD;
      ...&#xD;
   } &#xD;
&lt;/pre>&#xD;
&lt;/blockquote>&#xD;
&lt;p>&#xD;
    But that might be wrong. The way to discover whether it is or not is to try the two cases separately, even if you&#xD;
    really believe it will make no difference. (Your beliefs are really what you're testing.) Note that you might be&#xD;
    executing the &lt;font size=&quot;+0&quot;>then&lt;/font> case of the &lt;font size=&quot;+0&quot;>if&lt;/font> statement more than once for other&#xD;
    reasons. Why not try one of them with the result less than 0 and one with the result exactly equal to zero?&#xD;
&lt;/p>&#xD;
&lt;h4>&#xD;
    Uncaught exceptions&#xD;
&lt;/h4>&#xD;
&lt;p>&#xD;
    Exceptions are a kind of distinct result. By way of background, consider this code:&#xD;
&lt;/p>&#xD;
&lt;blockquote>&#xD;
&lt;pre>&#xD;
void process(Reader r) {&#xD;
   ...&#xD;
   try {&#xD;
      ...&#xD;
      int c = r.read();&#xD;
      ...&#xD;
   } catch (IOException e) {&#xD;
      ...&#xD;
   }&#xD;
}&#xD;
&lt;/pre>&#xD;
&lt;/blockquote>&#xD;
&lt;p>&#xD;
    You'd expect to check whether the handler code really does the right thing with a read failure. But suppose an&#xD;
    exception is explicitly unhandled. Instead, it's allowed to propagate upward through the code under test. In Java, that&#xD;
    might look like this:&#xD;
&lt;/p>&#xD;
&lt;blockquote>&#xD;
&lt;pre>&#xD;
void process(Reader r) &lt;font color=&quot;#ff0000&quot;>&#xD;
&lt;i>&#xD;
&lt;b>&#xD;
throws IOException&lt;/b>&lt;/i>&lt;/font> {&#xD;
    ...&#xD;
    int c = r.read();&#xD;
    ...&#xD;
}&#xD;
&lt;/pre>&#xD;
&lt;/blockquote>&#xD;
&lt;p>&#xD;
    This technique asks you to test that case &lt;i>even though&lt;/i> the code explicitly doesn't handle it. Why? Because of&#xD;
    this kind of fault:&#xD;
&lt;/p>&#xD;
&lt;blockquote>&#xD;
&lt;pre>&#xD;
void process(Reader r) throws IOException {&#xD;
    ...&#xD;
    &lt;font color=&quot;#ff0000&quot;>&#xD;
&lt;i>&#xD;
&lt;b>&#xD;
Tracker.hold(this);&lt;/b>&lt;/i>&lt;/font>&#xD;
    ...&#xD;
    int c = r.read();&#xD;
    ...&#xD;
    &lt;font color=&quot;#ff0000&quot;>&#xD;
&lt;i>&#xD;
&lt;b>&#xD;
Tracker.release(this);&lt;/b>&lt;/i>&lt;/font>&#xD;
    ...&#xD;
}&#xD;
&lt;/pre>&#xD;
&lt;/blockquote>&#xD;
&lt;p>&#xD;
    Here, the code affects global state (through &lt;font size=&quot;+0&quot;>Tracker.hold&lt;/font>). If the exception is thrown, &lt;font     size=&quot;+0&quot;>Tracker.release&lt;/font> will never be called.&#xD;
&lt;/p>&#xD;
&lt;p>&#xD;
    (Notice that the failure to release will probably have no obvious immediate consequences. The problem will most likely&#xD;
    not be visible until &lt;font size=&quot;+0&quot;>process&lt;/font> is called again, whereupon the attempt to &lt;font     size=&quot;+0&quot;>hold&lt;/font> the object for a second time will fail. A good article about such defects is Keith Stobie's &lt;a href=&quot;http://www.testingcraft.com/stobie-exceptions.pdf&quot; target=&quot;_blank&quot;>&quot;Testing for Exceptions&quot;&lt;/a>. &amp;nbsp; (&lt;a href=&quot;http://www.adobe.com/products/acrobat/alternate.html&quot; target=&quot;_blank&quot;>Get Adobe Reader&lt;/a>))&#xD;
&lt;/p>&#xD;
&lt;h3>&#xD;
    &lt;a id=&quot;UndiscoveredFaults&quot; name=&quot;UndiscoveredFaults&quot;>Undiscovered faults&lt;/a>&#xD;
&lt;/h3>&#xD;
&lt;p>&#xD;
    This particular technique does not address all defects associated with method calls. Here are two kinds that it's&#xD;
    unlikely to catch.&#xD;
&lt;/p>&#xD;
&lt;h4>&#xD;
    Incorrect arguments&#xD;
&lt;/h4>&#xD;
&lt;p>&#xD;
    Consider these two lines of C code, where the first line is wrong and the second line is correct.&#xD;
&lt;/p>&#xD;
&lt;blockquote>&#xD;
&lt;pre>&#xD;
... strncmp(s1, s2, strlen(s1)) ...&#xD;
... strncmp(s1, s2, strlen(&lt;font color=&quot;#ff0000&quot;>&#xD;
&lt;i>&#xD;
&lt;b>&#xD;
s2&lt;/b>&lt;/i>&lt;/font>)) ...&#xD;
&lt;/pre>&#xD;
&lt;/blockquote>&#xD;
&lt;p>&#xD;
    &lt;font size=&quot;+0&quot;>strncmp&lt;/font> compares two strings and returns a number less than 0 if the first one is&#xD;
    lexicographically less than the second (would come earlier in a dictionary), 0 if they're equal, and a number greater&#xD;
    than 0 if the first one is lexicographically larger. However, it only compares the number of characters given by the&#xD;
    third argument. The problem is that the length of the first string is used to limit the comparison, whereas it should&#xD;
    be the length of the second.&#xD;
&lt;/p>&#xD;
&lt;p>&#xD;
    This technique would require three tests, one for each distinct return value. Here are three you could use:&#xD;
&lt;/p>&#xD;
&lt;div align=&quot;center&quot;>&#xD;
    &lt;table     style=&quot;BORDER-RIGHT: rgb(128,128,128) 1px solid; BORDER-TOP: rgb(128,128,128) 1px solid; BORDER-LEFT: rgb(128,128,128) 1px solid; BORDER-BOTTOM: rgb(128,128,128) 1px solid&quot;      cellspacing=&quot;0&quot; bordercolordark=&quot;#808080&quot; cellpadding=&quot;4&quot; width=&quot;85%&quot; bordercolorlight=&quot;#808080&quot; border=&quot;1&quot;>&#xD;
        &lt;tbody>&#xD;
            &lt;tr>&#xD;
                &lt;th scope=&quot;col&quot; align=&quot;middle&quot; width=&quot;25%&quot; bgcolor=&quot;#c0c0c0&quot;>&#xD;
                    s1&#xD;
                &lt;/th>&#xD;
                &lt;th scope=&quot;col&quot; align=&quot;middle&quot; width=&quot;25%&quot; bgcolor=&quot;#c0c0c0&quot;>&#xD;
                    s2&#xD;
                &lt;/th>&#xD;
                &lt;th scope=&quot;col&quot; align=&quot;middle&quot; width=&quot;25%&quot; bgcolor=&quot;#c0c0c0&quot;>&#xD;
                    expected result&#xD;
                &lt;/th>&#xD;
                &lt;th scope=&quot;col&quot; align=&quot;middle&quot; width=&quot;25%&quot; bgcolor=&quot;#c0c0c0&quot;>&#xD;
                    actual result&#xD;
                &lt;/th>&#xD;
            &lt;/tr>&#xD;
            &lt;tr>&#xD;
                &lt;td align=&quot;middle&quot; width=&quot;25%&quot;>&#xD;
                    &quot;a&quot;&#xD;
                &lt;/td>&#xD;
                &lt;td align=&quot;middle&quot; width=&quot;25%&quot;>&#xD;
                    &quot;bbb&quot;&#xD;
                &lt;/td>&#xD;
                &lt;td align=&quot;middle&quot; width=&quot;25%&quot;>&#xD;
                    &amp;lt;0&#xD;
                &lt;/td>&#xD;
                &lt;td align=&quot;middle&quot; width=&quot;25%&quot;>&#xD;
                    &amp;lt;0&#xD;
                &lt;/td>&#xD;
            &lt;/tr>&#xD;
            &lt;tr>&#xD;
                &lt;td align=&quot;middle&quot; width=&quot;25%&quot;>&#xD;
                    &quot;bbb&quot;&#xD;
                &lt;/td>&#xD;
                &lt;td align=&quot;middle&quot; width=&quot;25%&quot;>&#xD;
                    &quot;a&quot;&#xD;
                &lt;/td>&#xD;
                &lt;td align=&quot;middle&quot; width=&quot;25%&quot;>&#xD;
                    &amp;gt;0&#xD;
                &lt;/td>&#xD;
                &lt;td align=&quot;middle&quot; width=&quot;25%&quot;>&#xD;
                    &amp;gt;0&#xD;
                &lt;/td>&#xD;
            &lt;/tr>&#xD;
            &lt;tr>&#xD;
                &lt;td align=&quot;middle&quot; width=&quot;25%&quot;>&#xD;
                    &quot;foo&quot;&#xD;
                &lt;/td>&#xD;
                &lt;td align=&quot;middle&quot; width=&quot;25%&quot;>&#xD;
                    &quot;foo&quot;&#xD;
                &lt;/td>&#xD;
                &lt;td align=&quot;middle&quot; width=&quot;25%&quot;>&#xD;
                    =0&#xD;
                &lt;/td>&#xD;
                &lt;td align=&quot;middle&quot; width=&quot;25%&quot;>&#xD;
                    =0&#xD;
                &lt;/td>&#xD;
            &lt;/tr>&#xD;
        &lt;/tbody>&#xD;
    &lt;/table>&lt;br />&#xD;
&lt;/div>&#xD;
&lt;p>&#xD;
    The defect is not discovered because nothing in this technique &lt;i>forces&lt;/i> the third argument to have any particular&#xD;
    value. What's needed is a test case like this:&#xD;
&lt;/p>&#xD;
&lt;div align=&quot;center&quot;>&#xD;
    &lt;table     style=&quot;BORDER-RIGHT: rgb(128,128,128) 1px solid; BORDER-TOP: rgb(128,128,128) 1px solid; BORDER-LEFT: rgb(128,128,128) 1px solid; BORDER-BOTTOM: rgb(128,128,128) 1px solid&quot;      cellspacing=&quot;0&quot; bordercolordark=&quot;#808080&quot; cellpadding=&quot;4&quot; width=&quot;85%&quot; bordercolorlight=&quot;#808080&quot; border=&quot;1&quot;>&#xD;
        &lt;tbody>&#xD;
            &lt;tr>&#xD;
                &lt;th scope=&quot;col&quot; align=&quot;middle&quot; width=&quot;25%&quot; bgcolor=&quot;#c0c0c0&quot;>&#xD;
                    &lt;b>s1&lt;/b>&#xD;
                &lt;/th>&#xD;
                &lt;th scope=&quot;col&quot; align=&quot;middle&quot; width=&quot;25%&quot; bgcolor=&quot;#c0c0c0&quot;>&#xD;
                    &lt;b>s2&lt;/b>&#xD;
                &lt;/th>&#xD;
                &lt;th scope=&quot;col&quot; align=&quot;middle&quot; width=&quot;25%&quot; bgcolor=&quot;#c0c0c0&quot;>&#xD;
                    &lt;b>expected result&lt;/b>&#xD;
                &lt;/th>&#xD;
                &lt;th scope=&quot;col&quot; align=&quot;middle&quot; width=&quot;25%&quot; bgcolor=&quot;#c0c0c0&quot;>&#xD;
                    &lt;b>actual result&lt;/b>&#xD;
                &lt;/th>&#xD;
            &lt;/tr>&#xD;
            &lt;tr>&#xD;
                &lt;td align=&quot;middle&quot; width=&quot;25%&quot;>&#xD;
                    &quot;foo&quot;&#xD;
                &lt;/td>&#xD;
                &lt;td align=&quot;middle&quot; width=&quot;25%&quot;>&#xD;
                    &quot;foo&lt;font color=&quot;#ff0000&quot;>&lt;i>&lt;b>d&lt;/b>&lt;/i>&lt;/font>&quot;&#xD;
                &lt;/td>&#xD;
                &lt;td align=&quot;middle&quot; width=&quot;25%&quot;>&#xD;
                    &lt;font color=&quot;#ff0000&quot;>&lt;i>&lt;b>&amp;lt;0&lt;/b>&lt;/i>&lt;/font>&#xD;
                &lt;/td>&#xD;
                &lt;td align=&quot;middle&quot; width=&quot;25%&quot;>&#xD;
                    =0&#xD;
                &lt;/td>&#xD;
            &lt;/tr>&#xD;
        &lt;/tbody>&#xD;
    &lt;/table>&lt;br />&#xD;
&lt;/div>&#xD;
&lt;p>&#xD;
    While there are techniques suitable for catching such defects, they are seldom used in practice. Your testing effort is&#xD;
    probably better spent on a rich set of tests that targets many types of defects (and that you hope catches this type as&#xD;
    a side effect).&#xD;
&lt;/p>&#xD;
&lt;h4>&#xD;
    Indistinct results&#xD;
&lt;/h4>&#xD;
&lt;p>&#xD;
    There's a danger that comes when you're coding - and testing - method-by-method. Here's an example. There are two&#xD;
    methods. The first, &lt;font size=&quot;+0&quot;>connect&lt;/font>, wants to establish a network connection:&#xD;
&lt;/p>&#xD;
&lt;blockquote>&#xD;
&lt;pre>&#xD;
void connect() {&#xD;
   ...&#xD;
   Integer portNumber = serverPortFromUser();&#xD;
   if (portNumber == null) {&#xD;
      // pop up message about invalid port number&#xD;
      return;&#xD;
   }&#xD;
&lt;/pre>&#xD;
&lt;/blockquote>&#xD;
&lt;p>&#xD;
    It calls &lt;font size=&quot;+0&quot;>serverPortFromUser&lt;/font> to get a port number. That method returns two distinct values. It&#xD;
    returns a port number chosen by the user if the number chosen is valid (1000 or greater). Otherwise, it returns null.&#xD;
    If null is returned, the code under test pops up an error message and quits.&#xD;
&lt;/p>&#xD;
&lt;p>&#xD;
    When &lt;font size=&quot;+0&quot;>connect&lt;/font> was tested, it worked as intended: a valid port number caused a connection to be&#xD;
    established, and an invalid one led to a popup.&#xD;
&lt;/p>&#xD;
&lt;p>&#xD;
    The code to &lt;font size=&quot;+0&quot;>serverPortFromUser&lt;/font> is a bit more complicated. It first pops up a window that asks&#xD;
    for a string and has the standard OK and CANCEL buttons. Based on what the user does, there are four cases:&#xD;
&lt;/p>&#xD;
&lt;ol>&#xD;
    &lt;li>&#xD;
        If the user types a valid number, that number is returned.&#xD;
    &lt;/li>&#xD;
    &lt;li>&#xD;
        If the number is too small (less than 1000), null is returned (so the message about invalid port number will be&#xD;
        displayed).&#xD;
    &lt;/li>&#xD;
    &lt;li>&#xD;
        If the number is misformatted, null is again returned (and the same message is appropriate).&#xD;
    &lt;/li>&#xD;
    &lt;li>&#xD;
        If the user clicks CANCEL, null is returned.&#xD;
    &lt;/li>&#xD;
&lt;/ol>&#xD;
&lt;p>&#xD;
    This code also works as intended.&#xD;
&lt;/p>&#xD;
&lt;p>&#xD;
    The combination of the two chunks of code, though, has a bad consequence: the user presses CANCEL and gets a message&#xD;
    about an invalid port number. All the code works as intended, but the overall effect is still wrong. It was tested in a&#xD;
    reasonable way, but a defect was missed.&#xD;
&lt;/p>&#xD;
&lt;p>&#xD;
    The problem here is that &lt;font size=&quot;+0&quot;>null&lt;/font> is one result that represents two distinct &lt;i>meanings&lt;/i> (&quot;bad&#xD;
    value&quot; and &quot;user cancelled&quot;). Nothing in this technique forces you to notice that problem with the design of &lt;font     size=&quot;+0&quot;>serverPortFromUser&lt;/font>.&#xD;
&lt;/p>&#xD;
&lt;p>&#xD;
    Testing can help, though. When &lt;font size=&quot;+0&quot;>serverPortFromUser&lt;/font> is tested in isolation - just to see if it&#xD;
    returns the intended value in each of those four cases - the context of use is lost. Instead, suppose it were tested&#xD;
    via &lt;font size=&quot;+0&quot;>connect&lt;/font>. There would be four tests that would exercise both of the methods simultaneously:&#xD;
&lt;/p>&#xD;
&lt;div align=&quot;center&quot;>&#xD;
    &lt;table     style=&quot;BORDER-RIGHT: rgb(128,128,128) 1px solid; BORDER-TOP: rgb(128,128,128) 1px solid; BORDER-LEFT: rgb(128,128,128) 1px solid; BORDER-BOTTOM: rgb(128,128,128) 1px solid&quot;      cellspacing=&quot;0&quot; bordercolordark=&quot;#808080&quot; cellpadding=&quot;4&quot; width=&quot;85%&quot; bordercolorlight=&quot;#808080&quot; border=&quot;1&quot;>&#xD;
        &lt;tbody>&#xD;
            &lt;tr>&#xD;
                &lt;th scope=&quot;col&quot; align=&quot;middle&quot; width=&quot;25%&quot; bgcolor=&quot;#c0c0c0&quot;>&#xD;
                    input&#xD;
                &lt;/th>&#xD;
                &lt;th scope=&quot;col&quot; align=&quot;middle&quot; width=&quot;25%&quot; bgcolor=&quot;#c0c0c0&quot;>&#xD;
                    expected result&#xD;
                &lt;/th>&#xD;
                &lt;th scope=&quot;col&quot; align=&quot;middle&quot; width=&quot;25%&quot; bgcolor=&quot;#c0c0c0&quot;>&#xD;
                    thought process&#xD;
                &lt;/th>&#xD;
            &lt;/tr>&#xD;
            &lt;tr>&#xD;
                &lt;td align=&quot;middle&quot; width=&quot;25%&quot;>&#xD;
                    user types &quot;1000&quot;&#xD;
                &lt;/td>&#xD;
                &lt;td align=&quot;middle&quot; width=&quot;25%&quot;>&#xD;
                    connection to port 1000 is opened&#xD;
                &lt;/td>&#xD;
                &lt;td align=&quot;middle&quot; width=&quot;25%&quot;>&#xD;
                    &lt;font size=&quot;+0&quot;>serverPortFromUser&lt;/font> returns a number, which is used.&#xD;
                &lt;/td>&#xD;
            &lt;/tr>&#xD;
            &lt;tr>&#xD;
                &lt;td>&#xD;
                    &lt;p align=&quot;center&quot;>&#xD;
                        user types &quot;999&quot;&#xD;
                    &lt;/p>&#xD;
                &lt;/td>&#xD;
                &lt;td>&#xD;
                    &lt;p align=&quot;center&quot;>&#xD;
                        popup about invalid port number&#xD;
                    &lt;/p>&#xD;
                &lt;/td>&#xD;
                &lt;td>&#xD;
                    &lt;p align=&quot;center&quot;>&#xD;
                        &lt;font size=&quot;+0&quot;>serverPortFromUser&lt;/font> returns null, which leads to popup&#xD;
                    &lt;/p>&#xD;
                &lt;/td>&#xD;
            &lt;/tr>&#xD;
            &lt;tr>&#xD;
                &lt;td align=&quot;middle&quot; width=&quot;25%&quot;>&#xD;
                    &lt;p align=&quot;center&quot;>&#xD;
                        user types &quot;i99&quot;&#xD;
                    &lt;/p>&#xD;
                &lt;/td>&#xD;
                &lt;td align=&quot;middle&quot; width=&quot;25%&quot;>&#xD;
                    popup about invalid port number&#xD;
                &lt;/td>&#xD;
                &lt;td align=&quot;middle&quot; width=&quot;25%&quot;>&#xD;
                    &lt;font size=&quot;+0&quot;>serverPortFromUser&lt;/font> returns null, which leads to popup&#xD;
                &lt;/td>&#xD;
            &lt;/tr>&#xD;
            &lt;tr>&#xD;
                &lt;td align=&quot;middle&quot; width=&quot;25%&quot;>&#xD;
                    users clicks CANCEL&#xD;
                &lt;/td>&#xD;
                &lt;td align=&quot;middle&quot; width=&quot;25%&quot;>&#xD;
                    whole connection process should be cancelled&#xD;
                &lt;/td>&#xD;
                &lt;td align=&quot;middle&quot; width=&quot;25%&quot;>&#xD;
                    &lt;font size=&quot;+0&quot;>&lt;i>serverPortFromUser&lt;/i>&lt;/font> &lt;i>returns null, hey wait a minute that doesn't make&#xD;
                    sense...&lt;/i>&#xD;
                &lt;/td>&#xD;
            &lt;/tr>&#xD;
        &lt;/tbody>&#xD;
    &lt;/table>&lt;br />&#xD;
&lt;/div>&#xD;
&lt;p>&#xD;
    As is often the case, testing in a larger context reveals integration problems that escape small-scale testing. And, as&#xD;
    is also often the case, careful thought during test design reveals the problem before the test is run. (But if the&#xD;
    defect isn't caught then, it will be caught when the test is run.)&lt;br />&#xD;
    &lt;br />&#xD;
&lt;/p></mainDescription>
</org.eclipse.epf.uma:ContentDescription>