| <?xml version="1.0" encoding="UTF-8"?> |
| <org.eclipse.epf.uma:ContentDescription xmi:version="2.0" |
| xmlns:xmi="http://www.omg.org/XMI" xmlns:org.eclipse.epf.uma="http://www.eclipse.org/epf/uma/1.0.5/uma.ecore" |
| xmlns:epf="http://www.eclipse.org/epf" epf:version="1.5.0" xmi:id="_vuwC4MPcEdmbOvqy4O0adg" |
| name="programming_automated_tests,_0j5sUMlgEdmt3adZL5Dmdw" guid="_vuwC4MPcEdmbOvqy4O0adg" |
| changeDate="2006-12-07T13:06:38.445-0800" version="1.0.0"> |
| <mainDescription><h3>
 |
| Introduction
 |
| </h3>
 |
| <p>
 |
| Although the programming of automated tests should contribute to the overall test effort, it usually does not make up
 |
| the entire test effort. In fact, test environments that are based on a complete automation approach end up spending
 |
| more time on test automation than on testing. Before you begin developing automated test scripts, consider first
 |
| whether it is more efficient to perform manual testing. Some aspects of an application are more efficiently tested
 |
| manually (for example, GUI testing versus data-drive testing). If you decide to program automated test scripts, examine
 |
| what aspects of your test scripting can be automated and begin designing your scripts.
 |
| </p>
 |
| <h3>
 |
| Design your automated tests
 |
| </h3>
 |
| <p>
 |
| Without some level of design of your automated tests, introducing automation into your testing effort can lead to more
 |
| problems than it solves. You should consider developing your automated tests according to a lifecycle with automation
 |
| test requirements, design, testing of the automation tests, and implementation of the automation tests. This approach
 |
| can be informal or formal depending on your project needs. By designing the programming of your automated tests, you
 |
| can avoid spending time programming the wrong tests, re-working programmed tests, deciphering different coding styles
 |
| in the programming of the tests, etc.
 |
| </p>
 |
| <h3>
 |
| Recorded versus programmed scripts
 |
| </h3>
 |
| <p>
 |
| Although there are clear benefits to recorded scripts (for example, ease of creation or ability for novice testers to
 |
| learn a scripting language), recorded scripts also present their own problems. The disadvantages of playback scripts
 |
| are well known. They are deceptively easy to create but very difficult to update. Problems with script reliability,
 |
| hard-coded data values, or changes to the application under test and the need to re-record are well-documented. On the
 |
| other hand, programming scripts can present difficulties of their own: they are difficult for the novice tester to
 |
| create, they can require substantial time and effort to develop, and they can be difficult to debug. Most test tooling
 |
| makes these issues less problematic by providing the tester script support functions, such as ways to establish target
 |
| of test lists, systematic ways to program verification point, point to datapools, build commands into the script (for
 |
| example, sleeper commands), comment the script, and document the script. Another major advantage, which is often
 |
| overlooked, of using testing tooling to mitigate these risks is the ability to add to an existing script in the form of
 |
| making corrections to an existing script, testing new features of a test target or application under test, or resuming
 |
| a recording after an interruption.
 |
| </p>
 |
| <h3>
 |
| Functional and performance test scripts
 |
| </h3>
 |
| <p>
 |
| When discussing automating test scripts, it is important to distinguish between functional and performance tests. Most
 |
| discussions of programming automated test scripts focus on testing the functionality of an application. This is not
 |
| inappropriate, since a lot of automated testing focuses on functional testing. Performance test scripting, however, has
 |
| its unique characteristics. Performance test automation provides you with the ability to programmatically set workloads
 |
| by adding user groups to test loads under group usage, setting think time behavior, running tests randomly or at set
 |
| rates, or setting the duration of a run. Performance test automation also allows you to create and maintain schedules
 |
| for your tests.
 |
| </p>
 |
| <h3>
 |
| Testing test scripts
 |
| </h3>
 |
| <p>
 |
| When testing your test scripts, keep in mind whether you are testing recorded or programmed test scripts. For recorded
 |
| scripts, much of the debugging of the script consists of errors that are introduced due to changes in the test target
 |
| or test environment. When you run a recorded test script, consider the test target of the script. Some test automation
 |
| tools capture this information as a part of the test script. Debugging a recorded script consists largely of
 |
| determining whether changes in the target have created error conditions in the script. In general, there are two main
 |
| categories to examine here: changes in the UI and test session sensitive data (for example, date stamped data). In most
 |
| cases, discrepancies between recording and playback cause errors in your recorded test scripts.
 |
| </p>
 |
| <p>
 |
| Testing programmed test scripts involves many of the same debugging techniques you would apply to debugging an
 |
| application. Consider both the flow control logic and the data aspects of your script. Automated testing tools provide
 |
| you with test script debugging IDEs as well as datapool management features that facilitate this type of testing.
 |
| During execution of test scripts, a test that uses a datapool can replace values in the programmed test with variable
 |
| test data that is stored in the datapool.
 |
| </p></mainDescription> |
| </org.eclipse.epf.uma:ContentDescription> |
