| <?xml version="1.0" encoding="UTF-8"?> |
| <org.eclipse.epf.uma:TaskDescription xmi:version="2.0" xmlns:xmi="http://www.omg.org/XMI" xmlns:org.eclipse.epf.uma="http://www.eclipse.org/epf/uma/1.0.3/uma.ecore" epf:version="1.0.0" xmi:id="_NrbRUqeqEdmKDbQuyzCoqQ" name="run_tests,_0jVEkMlgEdmt3adZL5Dmdw" guid="_NrbRUqeqEdmKDbQuyzCoqQ" changeDate="2006-12-07T16:24:24.839-0500" version="1.0.0"> |
| <mainDescription>Run the system test, which addresses functional and system integration tests and, potentially, user acceptance tests.</mainDescription> |
| <keyConsiderations><ul> |
| <li> |
| Run the test regularly. Ideally, that means whenever the code changes but, minimally, once a day. |
| </li> |
| <li> |
| It should be possible for anyone on the test team to run the test at any time. |
| </li> |
| </ul></keyConsiderations> |
| <sections xmi:id="_fR4aQKuSEdmhFZtkg1nakg" name="Schedule test execution" guid="_fR4aQKuSEdmhFZtkg1nakg"> |
| <sectionDescription><p> |
| Run&nbsp;the system tests as often as possible. Ideally, run&nbsp;the tests whenever new code is checked into&nbsp;the |
| version control tool. |
| </p> |
| <p> |
| For larger systems, this will be too expensive.&nbsp;The tests may take several hours to run; therefore, you'll need to |
| schedule tests less frequently. If possible, however, run the tests several times a day. As a minimum, |
| run&nbsp;automated tests each night. |
| </p></sectionDescription> |
| </sections> |
| <sections xmi:id="_gV408KuSEdmhFZtkg1nakg" name="Run the test" guid="_gV408KuSEdmhFZtkg1nakg"> |
| <sectionDescription><p> |
| Run the test at the scheduled time based on the instructions in the <a class="elementLink" href="./../../openup_basic/workproducts/test_script,_0ZfMEMlgEdmt3adZL5Dmdw.html" guid="_0ZfMEMlgEdmt3adZL5Dmdw">Test Script</a>. It is best that the script&nbsp;be automated. |
| </p> |
| <p> |
| Good practices: |
| </p> |
| <ol> |
| <li> |
| Run the test in a separate test environment. |
| </li> |
| <li> |
| Ensure that you run the test against the latest clean build. |
| </li> |
| <li> |
| The first step of the test should be to set up the test environment (ensure that the network is available, that the |
| test database is available and reset to a known state, and so on). |
| </li> |
| </ol></sectionDescription> |
| </sections> |
| <sections xmi:id="_hfVJQKuSEdmhFZtkg1nakg" name="Close test run" guid="_hfVJQKuSEdmhFZtkg1nakg"> |
| <sectionDescription>Close the actual run as the last step of running the test.&nbsp;To do this: |
| <ol> |
| <li> |
| Close the test logs. The&nbsp;test log files should be closed and placed in the appropriate folder or directory. |
| </li> |
| <li> |
| Announce results. Send a notice to everyone involved in the project informing them of the result of the test run |
| and where they can find the test logs. |
| </li> |
| </ol></sectionDescription> |
| </sections> |
| <sections xmi:id="_sQaC4DO2EduqsLmIADMQ9g" name="Examine the test log" guid="_sQaC4DO2EduqsLmIADMQ9g"> |
| <sectionDescription><p> |
| Collect and compile information from test execution logs so you can: |
| </p> |
| <ul> |
| <li> |
| Capture the high-impact and risk issues discovered in running the tests. |
| </li> |
| <li> |
| Identify errors in test creation, data inputs, or integrating applications and any architectural anomalies. |
| </li> |
| <li> |
| Isolate the target of the test to determine failure points. |
| </li> |
| <li> |
| Diagnose failure symptoms and characteristics. |
| </li> |
| <li> |
| Assess and identify possible solutions. |
| </li> |
| </ul> |
| <p> |
| After completing these steps, verify that you have enough details to determine the impact of the results. In addition, |
| make sure that enough information exists to assist individuals who are performing dependent tasks. |
| </p></sectionDescription> |
| </sections> |
| <sections xmi:id="_0XzAwDO2EduqsLmIADMQ9g" name="Identify failures and propose solutions" guid="_0XzAwDO2EduqsLmIADMQ9g"> |
| <sectionDescription><p> |
| Identify whether or not the test has failed and propose a solution based on the type of test and category of |
| failure.&nbsp; The approach to testing will determine the identified failures and candidates for solutions. |
| </p> |
| <p> |
| Tests that are programmer supporting are used to help prepare and ensure confidence in the code.&nbsp;When identifying |
| failures and proposing solutions for programmer supporting tests: |
| </p> |
| <ul> |
| <li> |
| Failures will be identified at an object or element level. |
| </li> |
| <li> |
| Solutions will be to help clarify the problem. |
| </li> |
| </ul> |
| <p> |
| Test that are business supporting are used to uncover prior mistakes and omissions. |
| </p> |
| <ul> |
| <li> |
| Failures will identify omissions in requirements. |
| </li> |
| <li> |
| Solutions will help to clarify expectations of the system. |
| </li> |
| </ul> |
| <p> |
| After you have this information and the steps proposed to resolve the failures, you can effectively categorize the type |
| of failure and the appropriate type of solution. |
| </p> |
| <p> |
| See <a class="elementLinkWithUserText" href="./../../openup_basic/guidances/supportingmaterials/references,_9ToeIB83Edqsvps02rpOOg.html" guid="_9ToeIB83Edqsvps02rpOOg">[MAR03]</a> for more information. |
| </p></sectionDescription> |
| </sections> |
| <sections xmi:id="_3t6oADO2EduqsLmIADMQ9g" name="Communicate test results" guid="_3t6oADO2EduqsLmIADMQ9g"> |
| <sectionDescription><p> |
| Communicate the test results to the team. For failed tests this might involve adding bugs to the <a class="elementLink" href="./../../openup_basic/workproducts/work_items_list,_rGNWsCbSEdqh1LYUOGRh2A.html" guid="_rGNWsCbSEdqh1LYUOGRh2A">Work Items List</a>. |
| </p> |
| <p> |
| Communicating test results can affect the perception of the effectiveness of the tests. When communicating test |
| results, it is important that you: |
| </p> |
| <ul> |
| <li> |
| Know the audience, so that appropriate information is communicated appropriately |
| </li> |
| <li> |
| Run tests or scenarios that are likely to uncover the high-impact and risk issues or represent actual use of the |
| system |
| </li> |
| </ul> |
| <p> |
| When preparing test result reports, answer the following questions: |
| </p> |
| <ul> |
| <li> |
| How many test cases exist, and what are their states (pass, fail, blocked, and so on)? |
| </li> |
| <li> |
| How many bug reports have been filed, and what are their states (open, assigned, ready for testing, closed, |
| deferred)? |
| </li> |
| <li> |
| What trends and patterns do you see in test case and bug report states, especially opened and closed bug reports |
| and passed and failed test cases? |
| </li> |
| <li> |
| For test cases that were blocked or skipped, why are they in this state? |
| </li> |
| <li> |
| Considering all test cases not yet run (and perhaps not even created yet), what key risks and areas of |
| functionality remain untested? |
| </li> |
| <li> |
| For failed test cases, what are the associated bug reports? |
| </li> |
| <li> |
| For bug reports ready for confirmation testing, when can your team perform the test? |
| </li> |
| </ul></sectionDescription> |
| </sections> |
| <purpose>To execute tests and evaluate the test results.</purpose> |
| </org.eclipse.epf.uma:TaskDescription> |