| <?xml version="1.0" encoding="UTF-8"?> |
| <!DOCTYPE html PUBLIC "-//W3C/DTD XHTML 1.0 Strict//EN" "DTD/xhtml1-strict.dtd"> |
| <!-- VERSION rmc:7.1.0 --> |
| <html xmlns="http://www.w3.org/1999/xhtml"> |
| <head> |
| <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/> |
| <!-- START NON-TRANSLATABLE --> |
| <title>openup&#92;guidances&#92;concepts&#92;&#92;types_of_developer_tests.xmi</title> |
| </head> |
| <!-- WARNING: do not modify the generated comments in this file below this line. They are used as markers for the import process. --> |
| <body> |
| Element Name: types_of_developer_tests.xmi<br/><br/> |
| <!-- END NON-TRANSLATABLE --> |
| <br/><br/><br/> |
| <!-- START NON-TRANSLATABLE --> |
| Attribute: presentationName<br/><br/> |
| <!-- END NON-TRANSLATABLE --> |
| <!-- START:presentationName,_eRutgC5QEduVhuZHT5jKZQ CRC: 104200557 -->Types of Developer Tests<!-- END:presentationName,_eRutgC5QEduVhuZHT5jKZQ --> |
| <br/><br/><br/> |
| <!-- START NON-TRANSLATABLE --> |
| Attribute: briefDescription<br/><br/> |
| <!-- END NON-TRANSLATABLE --> |
| <!-- START:briefDescription,_eRutgC5QEduVhuZHT5jKZQ CRC: 1082763834 -->This guideline describes various types of developer tests.<!-- END:briefDescription,_eRutgC5QEduVhuZHT5jKZQ --> |
| <br/><br/><br/> |
| <!-- START NON-TRANSLATABLE --> |
| Attribute: mainDescription<br/><br/> |
| <!-- END NON-TRANSLATABLE --> |
| <!-- START:mainDescription,-VGT8iHGtQSiOUGitq1qmow CRC: 1340386311 --><p> |
| This guideline describes a number of types of tests. To perform these types of testing you need to define, and then |
| run, a series of tests against the source code. A developer test is a single test that needs to be performed. |
| </p> |
| <p> |
| It is valuable to augment automated tests with human readable test scripts in order to implement developer test |
| cases, scripts that include the information discussed below. A test script is the actual steps, sometimes either |
| written procedures to follow or the source code of a test. Developer test scripts are run against testing targets: |
| either one unit of source code, a more complex portion of your system (such as a component), or the entire system |
| itself to test some developer issue such as integration. |
| </p> |
| <h3> |
| Regression Testing |
| </h3> |
| <p> |
| Regression testing is the act of ensuring that changes to the code have not adversely affected existing |
| functionality. It is important to recognize that incremental development makes regression testing critical. Whenever |
| you release an application, you must ensure its previous functionality still works, and because you release |
| applications more often when taking the incremental approach, this means regression testing becomes that much more |
| important. Regression testing is the first thing you should be thinking about when testing for the following reasons: |
| </p> |
| <ol> |
| <li> |
| You want to be able to modify code and know that you can rerun your tests to see if you broke anything. |
| </li> |
| <li> |
| Existing users get very angry when things that previously worked don’t work anymore. |
| </li> |
| </ol> |
| <p> |
| Regression testing is fairly straightforward conceptually – you just need to run all of the previous test cases against |
| the new version of the code. Regression testing tools help immensely because they are designed with regression testing |
| in mind. However, there are potential challenges to regression testing: |
| </p> |
| <ul> |
| <li> |
| When you change your production, either to enhance it or to refactor it, you will need to rework existing test |
| cases coupled to that code. |
| </li> |
| <li> |
| If your updates affect only a component of the system, then potentially you only need to run the test cases that |
| affect this single component. Although this approach is a little risky because your changes may have had a greater |
| impact than you suspect, it does help to reduce both the time and cost of regression testing. |
| </li> |
| <li> |
| The more non-code artifacts that you decide to keep, the greater the effort to regression test your work and |
| therefore the greater the risk to your project because you are more likely to skimp on your testing efforts. |
| </li> |
| </ul> |
| <p> |
| Regression testing is critical to success as an agile developer. Many software developers use the xUnit family of open |
| source tools, such as <a href="http://www.junit.org/" target="_blank">JUnit</a> and <a href="http://www.vbunit.org/" target="_blank">VBUnit</a>, to test their code. The advantage of these tools is that they implement a testing framework |
| with which you can regression test all of your source code. Commercial testing tools are also viable options. |
| </p> |
| <h3> |
| Traditional Code Testing Techniques |
| </h3> |
| <p> |
| Although object and procedural technologies are different, several important testing concepts from the procedural world |
| are valid regardless of the underlying technology. These traditional testing techniques are: |
| </p> |
| <ul> |
| <li> |
| Black-box testing |
| </li> |
| <li> |
| Clear-box testing |
| </li> |
| <li> |
| Boundary-value testing |
| </li> |
| <li> |
| Coverage/Path testing |
| </li> |
| </ul> |
| <h4> |
| Black-Box Testing |
| </h4> |
| <p> |
| Black-box testing, also called interface testing, is a technique in which you create test cases based only on the |
| expected functionality of a method, class, or application without any knowledge of its internal workings. One way to |
| define black-box testing is that given input A you should obtain expected results B. |
| </p> |
| <p> |
| The goal of black-box testing is to ensure the system can do what it should be able to do, but not how it does it. For |
| example, if you invoke differenceInDays(June 30 2006, July 3 2006) the expected result should be three. The creation of |
| black-box tests is often driven by the requirements for the system. The basic idea is to look at the user |
| requirement and ask what needs to be done to show that the user requirement is met. |
| </p> |
| <p> |
| The primary advantage of black-box testing is that it enables you to prove that your application fulfills the |
| requirements defined for it. The primary disadvantage is that it does not show that the internals |
| of the system work (hence the need for clear-box testing). |
| </p> |
| <h4> |
| Clear-Box Testing |
| </h4> |
| <p> |
| Clear-box testing, also called white-box testing, is based on the idea that the program code can drive the |
| development of test cases. The basic concept is you look at the code, and then create test cases that exercise it. |
| For example, assume you have access to the source code for differenceInDays(). When you look at it, you see an IF |
| statement determines whether the two dates are in the same year. If they are in the same year then a simple |
| strategy based on Julian dates is used; if not then a more complex strategy is used. This indicates that you need |
| at least one test that uses dates from the same year and one from different years. By looking at the code, you are able |
| to determine new test cases to exercise the different logic paths within it. |
| </p> |
| <p> |
| The primary advantage of this concept is that it motivates you to create tests that exercise specific lines of |
| code. The disadvantages are that it does not ensure that your code fulfils the actual requirements (hence the |
| need for black-box testing) and that your testing code becomes highly coupled to your application code. |
| </p> |
| <h4> |
| Boundary-Value Testing |
| </h4> |
| <p> |
| This is based on the knowledge that you need to test your code to ensure it can handle unusual and extreme situations. |
| For example, boundary-value test cases differenceInDays() would include passing it the same date, two wildly different |
| dates, one date on the last day of the year and the second on the first day of the following year, and one date on |
| February 29th of a leap year. The basic idea is you want to look for limits defined either by your business rules or by |
| common sense, and then create test cases to test attribute values in and around those values. |
| </p> |
| <p> |
| The primary advantage of boundary value testing is that it motivates you to confirm that your program code is able to |
| handle “unusual” or “extreme” cases. |
| </p> |
| <h4> |
| Coverage and Path Testing |
| </h4> |
| <p> |
| Two critical “traditional” concepts are coverage and path testing. Coverage testing is a technique in which you create |
| a series of test cases designed to test all the code paths in your code. In many ways, coverage testing is simply a |
| collection of clear-box test cases that together exercise every line of code in your application at least once. Path |
| testing is a superset of coverage testing that ensures not only have all lines of code been tested, but all paths of |
| logic have also been tested. The main difference occurs when you have a method with more than one set of case |
| statements or nested IF statements: to determine the number of test cases with coverage testing you would count the |
| maximum number of paths between the sets of case/nested IF statements and, with path testing, you would multiply the |
| number of logic paths. |
| </p> |
| <h4> |
| Unit and Integration Testing |
| </h4> |
| <p> |
| Unit testing is the testing of an item, such as an operation, in isolation. For example, the tests defined so far for |
| differenceInDays() are all unit tests. Integration testing, on the other hand, is the testing of a collection of items |
| to validate that they work together. In the case of the data library/class, do the various functions work together? |
| Perhaps the differenceInDays() function has a side effect that causes the dayOfWeek() function to fail if |
| differenceInDays() is called first. Integration testing looks for problems like this. |
| </p> |
| <h3> |
| Object-Oriented Testing Techniques |
| </h3> |
| <p> |
| When testing systems built using object technology it is important to understand that your source code is composed of |
| several constructs, including methods (operations), classes, and inheritance. Therefore you need testing techniques |
| that reflect the fact that you have these constructs. These techniques are: |
| </p> |
| <ul> |
| <li> |
| Method testing |
| </li> |
| <li> |
| Class testing |
| </li> |
| <li> |
| Class-integration testing |
| </li> |
| <li> |
| Inheritance-regression testing |
| </li> |
| </ul> |
| <h4> |
| Method Testing |
| </h4> |
| <p> |
| Method testing is the act of ensuring that your methods (operations) perform as defined. In procedural testing this |
| would have been called function/procedure testing. Issues to address via method testing include: |
| </p> |
| <ul> |
| <li> |
| Ensure that your getter/setter methods work as intended |
| </li> |
| <li> |
| Ensure that each method returns the proper values, including error messages and exceptions |
| </li> |
| <li> |
| Validate the parameters being passed to a method |
| </li> |
| <li> |
| Ensure that a method does what it should |
| </li> |
| </ul> |
| <p> |
| The advantage of method testing is that it ensures that methods work well in isolation but it does not help to find |
| unintended side effects. |
| </p> |
| <h4> |
| Class Testing |
| </h4> |
| <p> |
| The main purpose of class testing is to test classes in isolation, and it is effectively the combination of traditional |
| unit testing and integration testing. It is unit testing because you are testing the class and its instances as single |
| units in isolation, but it is also integration testing because you need to verify the methods and attributes of the |
| class work together. The one assumption you need to make while writing “class tests” is that all other classes in the |
| system work. Issues to address via class testing include: |
| </p> |
| <ul> |
| <li> |
| Validate that the attributes of an object are initialized properly |
| </li> |
| <li> |
| Validate the invariants of the class |
| </li> |
| </ul> |
| <p> |
| The primary advantages of class testing are that it validates that the operations and properties of a class work |
| together and that the class works in isolation. However, it does not guarantee that a class works well with the rest of |
| your system. |
| </p> |
| <h4> |
| Class-integration Testing |
| </h4> |
| <p> |
| Also known as component testing, this technique addresses the issue of whether the classes in your system, or a |
| component of your system, work together properly. The relationships between classes can be used to drive the |
| development of class integration test cases. Issues to address via class-integration testing include: |
| </p> |
| <ul> |
| <li> |
| Validate that objects do what other objects expect of them |
| </li> |
| <li> |
| Validate that return values are acted appropriately |
| </li> |
| <li> |
| Validate that exceptions/errors are processed appropriately |
| </li> |
| </ul> |
| <p> |
| The technique helps to validate that the various classes within a component, or a system, work together. However, it |
| can be difficult to define and develop the test cases to fully perform this level of testing. |
| </p> |
| <h4> |
| Inheritance-regression Testing |
| </h4>This is the act of running of test cases defined for the superclasses on the instances of a subclass because errors |
| have not been introduced by that new subclass. New methods are added and existing methods may be redefined by subclasses, |
| methods which access and potentially change the value of the attributes defined in the superclass. It is possible that a |
| subclass may change the value of the attributes in a way that was never intended in the superclass, or at least was never |
| expected. The point is that you need to invoke the test suite of the superclass(es) when testing a subclass. <br /><!-- END:mainDescription,-VGT8iHGtQSiOUGitq1qmow --> |
| </body> |
| </html> |