Google Git
Sign in
eclipse / epf / org.eclipse.epf.libraries / 8350724ae264a0fa24a769d4684cddef5f01f957 / . / XP / xp / guidances / concepts / developer_testing.xmi
blob: eda062754200347c29ddf0bc2e2ceb3ad032117c [file] [log] [blame]
<?xml version="1.0" encoding="UTF-8"?>
<org.eclipse.epf.uma:ContentDescription xmi:version="2.0" xmlns:xmi="http://www.omg.org/XMI" xmlns:org.eclipse.epf.uma="http://www.eclipse.org/epf/uma/1.0.3/uma.ecore" epf:version="1.0.0" xmi:id="-aJBLg1aguP1bIWvQbJSd6w" name="developer_testing,4.085829182735815E-305" guid="-aJBLg1aguP1bIWvQbJSd6w" changeDate="2006-11-13T14:42:19.105-0800" version="1.0.0">
<mainDescription>&lt;a id=&quot;XE_test__developer_testing__concept_of&quot; name=&quot;XE_test__developer_testing__concept_of&quot;&gt;&lt;/a&gt;&lt;a
id=&quot;XE_design__developer_testing__concept_of&quot; name=&quot;XE_design__developer_testing__concept_of&quot;&gt;&lt;/a&gt;
&lt;h3&gt;
&lt;a id=&quot;Introduction&quot; name=&quot;Introduction&quot;&gt;&lt;/a&gt;Introduction
&lt;/h3&gt;
&lt;p&gt;
The phrase &quot;Developer Testing&quot; is used to categorize the testing activities most appropriately performed by software
developers. It also includes the artifacts created by those activities. Developer Testing encompasses the work
traditionally thought of under the following categories: Unit Testing, much of Integration Testing, and some aspects of
what is most often referred to as System Testing. While Developer Testing is traditionally associated with activities
in the Implementation discipline, it also has a relationship to activities in the Analysis and Design discipline.
&lt;/p&gt;
&lt;p&gt;
By thinking of Developer Testing in this &quot;holistic&quot; way, you help to mitigate some of the risk associated with the more
&quot;atomistic&quot; approach traditionally taken. In the traditional approach to Developer Testing, the effort is initially
focused on evaluating that all units are working independently. Late in the development life-cycle, as the development
work nears completion, the integrated units are assembled into a working subsystem or system and tested in this setting
for the first time.
&lt;/p&gt;
&lt;p&gt;
This approach has a number of failings. Firstly, because it encourages a staged approach to the testing of the
integrated units and later subsystems, any errors identified during these tests are often found too late. This late
discovery typically results in the decision to take no corrective action, or it requires major rework to correct. This
rework is both expensive and detracts from making forward progress in other areas. This increases the risk of the
project being derailed or abandoned.
&lt;/p&gt;
&lt;p&gt;
Secondly, creating rigid boundaries between Unit, Integration and System Test increases the probability that errors
spanning the boundaries will be discovered by no one. The risk is compounded when responsibility for these types of
tests is assigned to separate teams.
&lt;/p&gt;
&lt;p&gt;
The style of developer testing recommended by&amp;nbsp;iterative processes&amp;nbsp;encourages the developer to focus on the
most valuable and appropriate tests to conduct at the given point in time. Even within the scope of a single iteration,
it is usually more efficient for the developer to find and correct as many of the defects in her own code as possible,
without the additional overhead in hand-off to a separate test group. The desired result is the early discovery of the
most significant software errors&amp;nbsp;- regardless of whether those errors are in the independent unit, the integration
of the units or the working of the integrated units within a meaningful end-user scenario.
&lt;/p&gt;
&lt;h3&gt;
&lt;a id=&quot;DeveloperTestingPitfalls&quot; name=&quot;DeveloperTestingPitfalls&quot;&gt;&lt;/a&gt;Pitfalls Getting Started with Developer Testing
&lt;/h3&gt;
&lt;p&gt;
Many developers who begin trying to do a substantially more thorough job of testing give up the effort shortly
thereafter. They find that it does not seem to be yielding value. Further, some developers who begin well with
developer testing find that they've created an unmaintainable test suite that is eventually abandoned.
&lt;/p&gt;
&lt;p&gt;
This page gives some guidelines for getting over the first hurdles and for creating a test suite that avoids the
maintainability trap. For more information, see &lt;a href=&quot;../../workguid/wg_mnttstste.htm&quot;&gt;Guidelines: Maintaining
Automated Test Suites&lt;/a&gt;.
&lt;/p&gt;
&lt;h4&gt;
Establish expectations
&lt;/h4&gt;
&lt;p&gt;
Those who find developer testing rewarding do it. Those who view it as a chore find ways to avoid it. This is simply in
the nature of most developers in most industries, and treating it as a shameful lack of discipline hasn't historically
been successful. Therefore, as a developer you should expect testing to be rewarding and do what it takes to make it
rewarding.
&lt;/p&gt;
&lt;p&gt;
Ideal developer testing follows a very tight edit-test loop. You make a small change to the product, such as adding a
new method to a class, then you immediately rerun your tests. If any test breaks, you know exactly what code is the
cause. This easy, steady pace of development is the greatest reward of developer testing. A long debugging session
should be exceptional.
&lt;/p&gt;
&lt;p&gt;
Because it's not unusual for a change made in one class to break something in another, you should expect to rerun not
just the changed class's tests, but many tests. Ideally, you rerun the complete test suite for your component many
times per hour. Every time you make a significant change, you rerun the suite, watch the results, and either proceed to
the next change or fix the last change. Expect to spend some effort making that rapid feedback possible.
&lt;/p&gt;
&lt;h4&gt;
Automate your tests
&lt;/h4&gt;
&lt;p&gt;
Running tests often is not practical if tests are manual. For some components, automated tests are easy. An example
would be an in-memory database. It communicates to its clients through an API and has no other interface to the outside
world. Tests for it would look something like this:
&lt;/p&gt;
&lt;blockquote&gt;
&lt;pre&gt;
/* Check that elements can be added at most once. */
// Setup
Database db = new Database();
db.add(&quot;key1&quot;, &quot;value1&quot;);
// Test
boolean result = db.add(&quot;key1&quot;, &quot;another value&quot;);
expect(result == false);
&lt;/pre&gt;
&lt;/blockquote&gt;
&lt;p&gt;
The tests are different from ordinary client code in only one way: instead of believing the results of API calls, they
check. If the API makes client code easy to write, it makes test code easy to write. If the test code is &lt;i&gt;not&lt;/i&gt;
easy to write, you've received an early warning that the API could be improved. Test-first design is thus consistent
with the iterative processes' focus on addressing important risks early.
&lt;/p&gt;
&lt;p&gt;
The more tightly connected the component is to the outside world, however, the harder it will be to test. There are two
common cases: graphical user interfaces and back-end components.
&lt;/p&gt;
&lt;h5&gt;
Graphical user interfaces
&lt;/h5&gt;
&lt;p&gt;
Suppose the database in the example above receives its data via a callback from a user-interface object. The callback
is invoked when the user fills in some text fields and pushes a button. Testing this by manually filling in the fields
and pushing the button isn't something you want to do many times an hour. You must arrange a way to deliver the input
under programmatic control, typically by &quot;pushing&quot; the button in code.
&lt;/p&gt;
&lt;p&gt;
Pushing the button causes some code in the component to be executed. Most likely, that code changes the state of some
user-interface objects. So you must also arrange a way to query those objects programmatically.
&lt;/p&gt;
&lt;h5&gt;
Back-end components
&lt;/h5&gt;
&lt;p&gt;
Suppose the component under test doesn't implement a database. Instead, it's a wrapper around a real, on-disk database.
Testing against that real database might be difficult. It might be hard to install and configure. Licenses for it might
be expensive. The database might slow down the tests enough that you're not inclined to run them often. In such cases,
it's worthwhile to &quot;stub out&quot; the database with a simpler component that does just enough to support the tests.
&lt;/p&gt;
&lt;p&gt;
Stubs are also useful when a component that your component talks to isn't ready yet. You don't want your testing to
wait on someone else's code.
&lt;/p&gt;
&lt;p&gt;
For more information, see &lt;a href=&quot;../implemen/co_stubs.htm&quot;&gt;Concepts: Stubs&lt;/a&gt;.
&lt;/p&gt;
&lt;h4&gt;
Don't write your own tools
&lt;/h4&gt;
&lt;p&gt;
Developer testing seems pretty straightforward. You set up some objects, make a call through an API, check the result,
and announce a test failure if the results aren't as expected. It's also convenient to have some way to group tests so
that they can be run individually or as complete suites. Tools that support those requirements are called &lt;i&gt;test
frameworks&lt;/i&gt;.
&lt;/p&gt;
&lt;p&gt;
Developer testing &lt;b&gt;is&lt;/b&gt; straightforward, and the requirements for test frameworks are not complicated. If, however,
you yield to the temptation of writing your own test framework, you'll spend much more time tinkering with the
framework than you probably expect. There are many test frameworks available, both commercial and open source, and
there's no reason not to use one of those.
&lt;/p&gt;
&lt;h4&gt;
Do create support code
&lt;/h4&gt;
&lt;p&gt;
Test code tends to be repetitive. It's common to see sequences of code like this:
&lt;/p&gt;
&lt;blockquote&gt;
&lt;pre&gt;
// null name not allowed
retval = o.createName(&quot;&quot;);
expect(retval == null);
// leading spaces not allowed
retval = o.createName(&quot; l&quot;);
expect(retval == null);
// trailing spaces not allowed
retval = o.createName(&quot;name &quot;);
expect(retval == null);
// first character may not be numeric
retval = o.createName(&quot;5allpha&quot;);
expect(retval == null);
&lt;/pre&gt;
&lt;/blockquote&gt;
&lt;p&gt;
This code is created by copying one check, pasting it, then editing it to make another check.
&lt;/p&gt;
&lt;p&gt;
The danger here is twofold. If the interface changes, much editing will have to be done. (In more complicated cases, a
simple global replacement won't suffice.) Also, if the code is at all complicated, the intent of the test can be lost
amid all the text.
&lt;/p&gt;
&lt;p&gt;
When you find yourself repeating yourself, seriously consider factoring out the repetition into support code. Even
though the code above is a simple example, it's more readable and maintainable if written like this:
&lt;/p&gt;
&lt;blockquote&gt;
&lt;pre&gt;
void expectNameRejected(MyClass o, String s) {
Object retval = o.createName(s);
expect(retval == null);
}
...
// null name not allowed
expectNameRejected(o, &quot;&quot;);
// leading spaces not allowed.
expectNameRejected(o, &quot; l&quot;);
// trailing spaces not allowed.
expectNameRejected(o, &quot;name &quot;);
// first character may not be numeric.
expectNameRejected(o, &quot;5alpha&quot;);
&lt;/pre&gt;
&lt;/blockquote&gt;
&lt;p&gt;
Developers writing tests often err on the side of too much copying-and-pasting. If you suspect yourself of that
tendency, it's useful to consciously err in the other direction. Resolve that you will strip your code of all duplicate
text.
&lt;/p&gt;
&lt;h4&gt;
Write the tests first
&lt;/h4&gt;
&lt;p&gt;
Writing the tests after the code is a chore. The urge is to rush through it, to finish up and move on. Writing tests
before the code makes testing part of a positive feedback loop. As you implement more code, you see more tests passing
until finally all the tests pass and you're done. People who write tests first seem to be more successful, and it takes
no more time. For more on putting tests first, see &lt;a class=&quot;elementLinkWithType&quot;
href=&quot;./../../../xp/guidances/concepts/test-first_design,6.556259235358794E-306.html&quot;
guid=&quot;6.556259235358794E-306&quot;&gt;Concept: Test-first Design&lt;/a&gt;
&lt;/p&gt;
&lt;h4&gt;
Keep the tests understandable
&lt;/h4&gt;
&lt;p&gt;
You should expect that you, or someone else, will have to modify the tests later. A typical situation is that a later
iteration calls for a change to the component's behavior. As a simple example, suppose the component once declared a
square root method like this:
&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;
&lt;font size=&quot;+0&quot;&gt;double sqrt(double x);&lt;/font&gt;
&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;
In that version, a negative argument caused &lt;font size=&quot;+0&quot;&gt;sqrt&lt;/font&gt; to return NaN (&quot;not a number&quot; from the IEEE
754-1985 &lt;i&gt;Standard for Binary Floating-Point Arithmetic&lt;/i&gt;). In the new iteration, the square root method will
accept negative numbers and return a complex result:
&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;
&lt;font size=&quot;+0&quot;&gt;Complex sqrt(double x);&lt;/font&gt;
&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;
Old tests for &lt;font size=&quot;+0&quot;&gt;sqrt&lt;/font&gt; will have to change. That means understanding what they do, and updating them
so that they work with the new &lt;font size=&quot;+0&quot;&gt;sqrt&lt;/font&gt;. When updating tests, you must take care not to destroy
their bug-finding power. One way that sometimes happens is this:
&lt;/p&gt;
&lt;blockquote&gt;
&lt;pre&gt;
void testSQRT () {
// Update these tests for Complex
// when I have time -- bem
/*
double result = sqrt(0.0);
...
*/
}
&lt;/pre&gt;
&lt;/blockquote&gt;
&lt;p&gt;
Other ways are more subtle: the tests are changed so that they actually run, but they no longer test what they were
originally intended to test. The end result, over many iterations, can be a test suite that is too weak to catch many
bugs. This is sometimes called &quot;test suite decay&quot;. A decayed suite will be abandoned, because it's not worth the
upkeep.
&lt;/p&gt;
&lt;p&gt;
You can't maintain a test's bug-finding power unless it's clear what &lt;a class=&quot;elementLinkWithUserText&quot;
href=&quot;./../../../xp/guidances/concepts/test-ideas_list,8.834380241450745E-306.html#TestIdeas&quot;
guid=&quot;8.834380241450745E-306&quot;&gt;Test Ideas&lt;/a&gt; a test implements. Test code tends to be under-commented, even though it's
often harder to understand the &quot;why&quot; behind it than product code.
&lt;/p&gt;
&lt;p&gt;
Test suite decay is less likely in the direct tests for &lt;font size=&quot;+0&quot;&gt;sqrt&lt;/font&gt; than in indirect tests. There will
be code that calls &lt;font size=&quot;+0&quot;&gt;sqrt&lt;/font&gt;. That code will have tests. When &lt;font size=&quot;+0&quot;&gt;sqrt&lt;/font&gt; changes,
some of those tests will fail. The person who changes &lt;font size=&quot;+0&quot;&gt;sqrt&lt;/font&gt; will probably have to change those
tests. Because he's less familiar with them, and because their relationship to the change is less clear, he's more
likely to weaken them in the process of making them pass.
&lt;/p&gt;
&lt;p&gt;
When you're creating support code for tests (as urged above), be careful: the support code should clarify, not obscure,
the purpose of the tests that use it. A common complaint about object-oriented programs is that there's no one place
where anything's done. If you look at any one method, all you discover is that it forwards its work somewhere else.
Such a structure has advantages, but it makes it harder for new people to understand the code. Unless they make an
effort, their changes are likely to be incorrect or to make the code even more complicated and fragile. The same is
true of test code, except that later maintainers are even less likely to take due care. You must head off the problem
by writing understandable tests.
&lt;/p&gt;
&lt;h4&gt;
Match the test structure to the product structure
&lt;/h4&gt;
&lt;p&gt;
Suppose someone has inherited your component. They need to change a part of it. They may want to examine the old tests
to help them in their new design. They want to update the old tests before writing the code (test-first design).
&lt;/p&gt;
&lt;p&gt;
All those good intentions will go by the wayside if they can't find the appropriate tests. What they'll do is make the
change, see what tests fail, then fix those. That will contribute to test suite decay.
&lt;/p&gt;
&lt;p&gt;
For that reason, it's important that the test suite be well structured, and that the location of tests be predictable
from the structure of the product. Most usually, developers arrange tests in a parallel hierarchy, with one test class
per product class. So if someone is changing a class named &lt;font size=&quot;+0&quot;&gt;Log&lt;/font&gt;, they know the test class is
&lt;font size=&quot;+0&quot;&gt;TestLog&lt;/font&gt;, and they know where the source file can be found.
&lt;/p&gt;
&lt;h4&gt;
Let tests violate encapsulation
&lt;/h4&gt;
&lt;p&gt;
You might limit your tests to interacting with your component exactly as client code does, through the same interface
that client code uses. However, this has disadvantages. Suppose you're testing a simple class that maintains a doubly
linked list:
&lt;/p&gt;
&lt;p align=&quot;center&quot;&gt;
&lt;img height=&quot;46&quot; alt=&quot;&quot; src=&quot;resources/dvltst-img1.gif&quot; width=&quot;195&quot; /&gt;
&lt;/p&gt;
&lt;p class=&quot;picturetext&quot;&gt;
Fig1: Double-linked list
&lt;/p&gt;
&lt;p&gt;
In particular, you're testing the &lt;font size=&quot;+0&quot;&gt;DoublyLinkedList.insertBefore(Object existing, Object
newObject)&lt;/font&gt; method. In one of your tests, you want to insert an element in the middle of the list, then check if
it's been inserted successfully. The test uses the list above to create this updated list:
&lt;/p&gt;
&lt;p align=&quot;center&quot;&gt;
&lt;img height=&quot;46&quot; alt=&quot;&quot; src=&quot;resources/dvltst-img2.gif&quot; width=&quot;318&quot; /&gt;
&lt;/p&gt;
&lt;p class=&quot;picturetext&quot;&gt;
Fig2: Double-linked list - item inserted
&lt;/p&gt;
&lt;p&gt;
It checks the list correctness like this:
&lt;/p&gt;
&lt;blockquote&gt;
&lt;pre&gt;
// the list is now one longer.
expect(list.size()==3);
// the new element is in the correct position
expect(list.get(1)==m);
// check that other elements are still there.
expect(list.get(0)==a);
expect(list.get(2)==z);
&lt;/pre&gt;
&lt;/blockquote&gt;
&lt;p&gt;
That seems sufficient, but it's not. Suppose the list implementation is incorrect and backward pointers are not set
correctly. That is, suppose the updated list actually looks like this:
&lt;/p&gt;
&lt;p align=&quot;center&quot;&gt;
&lt;img height=&quot;73&quot; alt=&quot;&quot; src=&quot;resources/dvltst-img3.gif&quot; width=&quot;318&quot; /&gt;
&lt;/p&gt;
&lt;p class=&quot;picturetext&quot;&gt;
Fig3: Double-linked list - fault in implementation
&lt;/p&gt;
&lt;p&gt;
If &lt;font size=&quot;+0&quot;&gt;DoublyLinkedList.get(int index)&lt;/font&gt; traverses the list from the beginning to the end (likely),
the test would miss this failure. If the class provides &lt;font size=&quot;+0&quot;&gt;elementBefore&lt;/font&gt; and &lt;font
size=&quot;+0&quot;&gt;elementAfter&lt;/font&gt; methods, checking for such failures is straightforward:
&lt;/p&gt;
&lt;blockquote&gt;
&lt;pre&gt;
// Check that links were all updated
expect(list.elementAfter(a)==m);
expect(list.elementAfter(m)==z);
expect(list.elementBefore(z)==m); //this will fail
expect(list.elementBefore(m)==a);
&lt;/pre&gt;
&lt;/blockquote&gt;
&lt;p&gt;
But what if it doesn't provide those methods? You could devise more elaborate sequences of method calls that will fail
if the suspected defect is present. For example, this would work:
&lt;/p&gt;
&lt;blockquote&gt;
&lt;pre&gt;
// Check whether back-link from Z is correct.
list.insertBefore(z, x);
// If it was incorrectly not updated, X will have
// been inserted just after A.
expect(list.get(1)==m);
&lt;/pre&gt;
&lt;/blockquote&gt;
&lt;p&gt;
But such a test is more work to create and is likely to be significantly harder to maintain. (Unless you write good
comments, it will not be at all clear why the test is doing what it's doing.) There are two solutions:
&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
Add the &lt;font size=&quot;+0&quot;&gt;elementBefore&lt;/font&gt; and &lt;font size=&quot;+0&quot;&gt;elementAfter&lt;/font&gt; methods to the public
interface. But that effectively exposes the implementation to everyone and makes future change more difficult.
&lt;/li&gt;
&lt;li&gt;
Let the tests &quot;look under the hood&quot; and check pointers directly.
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;
The latter is usually the best solution, even for a simple class like &lt;font size=&quot;+0&quot;&gt;DoublyLinkedList&lt;/font&gt; and
especially for the more complex classes that occur in your products.
&lt;/p&gt;
&lt;p&gt;
Typically, tests are put in the same package as the class they test. They are given protected or friend access.
&lt;/p&gt;
&lt;h3&gt;
&lt;a id=&quot;TestDesignMistakes&quot; name=&quot;TestDesignMistakes&quot;&gt;&lt;/a&gt;Characteristic Test Design Mistakes
&lt;/h3&gt;
&lt;p&gt;
Each test exercises a component and checks for correct results. The design of the test-the inputs it uses and how it
checks for correctness-can be good at revealing defects, or it can inadvertently hide them. Here are some
characteristic test design mistakes.
&lt;/p&gt;
&lt;h4&gt;
Failure to specify expected results in advance
&lt;/h4&gt;
&lt;p&gt;
Suppose you're testing a component that converts XML into HTML. A temptation is to take some sample XML, run it through
the conversion, then look at the results in a browser. If the screen looks right, you &quot;bless&quot; the HTML by saving it as
the official expected results. Thereafter, a test compares the actual output of the conversion to the expected results.
&lt;/p&gt;
&lt;p&gt;
This is a dangerous practice. Even sophisticated computer users are used to believing what the computer does. You are
likely to overlook mistakes in the screen appearance. (Not to mention that browsers are quite tolerant of misformatted
HTML.) By making that incorrect HTML the official expected results, you make sure that the test can never find the
problem.
&lt;/p&gt;
&lt;p&gt;
It's less dangerous to doubly-check by looking directly at the HTML, but it's still dangerous. Because the output is
complicated, it will be easy to overlook errors. You'll find more defects if you write the expected output by hand
first.
&lt;/p&gt;
&lt;h4&gt;
Failure to check the background
&lt;/h4&gt;
&lt;p&gt;
Tests usually check that what should have been changed has been, but their creators often forget to check that what
should have been left alone has been left alone. For example, suppose a program is supposed to change the first 100
records in a file. It's a good idea to check that the 101&lt;sup&gt;st&lt;/sup&gt; hasn't been changed.
&lt;/p&gt;
&lt;p&gt;
In theory, you would check that nothing in the &quot;background&quot;-the entire file system, all of memory, everything reachable
through the network-has been left alone. In practice, you have to choose carefully what you can afford to check. But
it's important to make that choice.
&lt;/p&gt;
&lt;h4&gt;
Failure to check persistence
&lt;/h4&gt;
&lt;p&gt;
Just because the component tells you a change has been made, that doesn't mean it has actually been committed to the
database. You need to check the database via another route.
&lt;/p&gt;
&lt;h4&gt;
Failure to add variety
&lt;/h4&gt;
&lt;p&gt;
A test might be designed to check the effect of three fields in a database record, but many other fields need to be
filled in to execute the test. Testers will often use the same values over and over again for these &quot;irrelevant&quot;
fields. For example, they'll always use the name of their lover in a text field, or 999 in a numeric field.
&lt;/p&gt;
&lt;p&gt;
The problem is that sometimes what shouldn't matter actually does. Every so often, there's a bug that depends on some
obscure combination of unlikely inputs. If you always use the same inputs, you stand no chance of finding such bugs. If
you persistently vary inputs, you might. Quite often, it costs almost nothing to use a number different than 999 or to
use someone else's name. When varying the values used in tests costs almost nothing and it has some potential benefit,
then vary. (Note: It's unwise to use names of old lovers instead of your current one if your current lover works with
you.)
&lt;/p&gt;
&lt;p&gt;
Here's another benefit. One plausible fault is for the program to use field &lt;i&gt;X&lt;/i&gt; when it should have used field
&lt;i&gt;Y&lt;/i&gt;. If both fields contain &quot;Dawn&quot;, the fault can't be detected.
&lt;/p&gt;
&lt;h4&gt;
Failure to use realistic data
&lt;/h4&gt;
&lt;p&gt;
It's common to use made-up data in tests. That data is often unrealistically simple. For example, customer names might
be &quot;Mickey&quot;, &quot;Snoopy&quot;, and &quot;Donald&quot;. Because that data is different from what real users enter - for example, it's
characteristically shorter - it can miss defects real customers will see. For example, these one-word names wouldn't
detect that the code doesn't handle names with spaces.
&lt;/p&gt;
&lt;p&gt;
It's prudent to make a slight extra effort to use realistic data.
&lt;/p&gt;
&lt;h4&gt;
Failure to notice that the code does nothing at all
&lt;/h4&gt;
&lt;p&gt;
Suppose you initialize a database record to zero, run a calculation that should result in zero being stored in the
record, then check that the record is zero. What has your test demonstrated? The calculation might not have taken place
at all. Nothing might have been stored, and the test couldn't tell.
&lt;/p&gt;
&lt;p&gt;
That example sounds unlikely. But this same mistake can crop up in subtler ways. For example, you might write a test
for a complicated installer program. The test is intended to check that all temporary files are removed after a
successful installation. But, because of all the installer options, in that test, one particular temporary file wasn't
created. Sure enough, that's the one the program forgot to remove.
&lt;/p&gt;
&lt;h4&gt;
Failure to notice that the code does the wrong thing
&lt;/h4&gt;
&lt;p&gt;
Sometimes a program does the right thing for the wrong reasons. As a trivial example, consider this code:
&lt;/p&gt;
&lt;blockquote&gt;
&lt;pre&gt;
if (a &amp;lt; b &amp;amp;&amp;amp; c)
return 2 * x;
else
return x * x;
&lt;/pre&gt;
&lt;/blockquote&gt;
&lt;p&gt;
The logical expression is wrong, and you've written a test that causes it to evaluate incorrectly and take the wrong
branch. Unfortunately, purely by coincidence, the variable X has the value 2 in that test. So the result of the wrong
branch is accidentally correct - the same as the result the right branch would have given.
&lt;/p&gt;
&lt;p&gt;
For each expected result, you should ask if there's a plausible way in which that result could be gotten for the wrong
reason. While it's often impossible to know, sometimes it's not.
&lt;/p&gt;
&lt;br /&gt;
&lt;br /&gt;</mainDescription>
</org.eclipse.epf.uma:ContentDescription>