| <?xml version="1.0" encoding="UTF-8"?> |
| <org.eclipse.epf.uma:TaskDescription xmi:version="2.0" |
| xmlns:xmi="http://www.omg.org/XMI" xmlns:org.eclipse.epf.uma="http://www.eclipse.org/epf/uma/1.0.5/uma.ecore" |
| xmlns:rmc="http://www.ibm.com/rmc" rmc:version="7.5.0" xmlns:epf="http://www.eclipse.org/epf" |
| epf:version="1.5.0" xmi:id="-l9UMxPIZrGi8cIWuAJ4OBQ" |
| name=",_mDL5YGhkEd-FVuzXjnvIWA" guid="-l9UMxPIZrGi8cIWuAJ4OBQ" changeDate="2010-05-25T18:29:17.875-0700"> |
| <mainDescription><p>
 |
| The recommended approach is to consider a rule-set as a unit piece of functionality that gets exposed to the rest of
 |
| the application as a decision service (SOA pattern). Therefore it should be tested using the same strategy as for any
 |
| other function in the system. One criterion of rule testing lies in the level of user friendliness that should be built
 |
| into the test harness. Indeed, empowering business analysts with the ability to change the rules also means they should
 |
| be tested with their changes. Typically developers code fixtures and do automated testing using JUnit. Business
 |
| analysts should not write code, but should be able to put together fixtures to be tested against the rules.
 |
| </p>
 |
| <p>
 |
| How much effort should be spent on the rule test harness really depends on the application, and also importantly on the
 |
| IT-level of business analyst. Sometimes capturing test data in XML and having business analysts edit them is enough and
 |
| very cost effective. Sometimes business analysts will feel more comfortable with Excel spreadsheets, or with a full GUI
 |
| which would need a larger development effort to be facilitated.<br />
 |
| It is usually a good idea whenever possible to keep individual test case data in files that can be put under source
 |
| code management. This way the code base, the rules and the test cases can be kept in sync. This becomes important when
 |
| multiple versions of a business rule are being worked on at different stages such as active development stage, test
 |
| stage or production stage.<br />
 |
| BRMS testing fits perfectly into this strategy due to its ability to produce the same sort of test fixtures as the
 |
| other environments. Utilizing a tool such as JUnit as the test controller, test fixtures are stored in a test fixture
 |
| repository for which the JUnit can access.
 |
| </p>
 |
| <h5>
 |
| Testing Levels<br />
 |
| </h5>
 |
| <p>
 |
| The following levels of testing are usually applied during the application construction:<br />
 |
| </p>
 |
| <ul>
 |
| <li>
 |
| Unit Test: applied to test classes and other application elements as developers build them. In the context of a
 |
| BRMS, the goal of unit testing is to make sure that a rule is actually fired when expected, against the expect
 |
| object instances and performs the expected working memory updates.
 |
| </li>
 |
| <li>
 |
| Component Test: applied to test a group of elements (e.g. classes) which together represent a significant
 |
| application component. In the context of a BRMS, a component is typically a rule service.
 |
| </li>
 |
| <li>
 |
| System Test: applied to test the system performs the right business functions correctly.
 |
| </li>
 |
| <li>
 |
| Acceptance Test: applied to ensure users are satisfied with the system. The last stages of acceptance testing are
 |
| the alpha testing (testing performed internally by the company) and beta testing (testing performed by a limited
 |
| audience outside of the company).
 |
| </li>
 |
| <li>
 |
| The scope of this section is limited to unit-test (usually performed by rule developers) and component test
 |
| (performed by both rule developers and rule authors), where components are understood as the rule services
 |
| implemented by a rule engine.
 |
| </li>
 |
| </ul>
 |
| <p>
 |
| Once the rule service components are tested, the next levels can follow the general application testing process
 |
| followed by the client's QA department, who are considering the rule services as black boxes.<br />
 |
| </p>
 |
| <h5>
 |
| Testing Activities
 |
| </h5>
 |
| <p>
 |
| This section relates to why testing is performed, that is what we are trying to get out of it. The goal of testing the
 |
| decision service components is first and foremost to ensure the correctness of the decision rendered by the service
 |
| with respect to the requirements.<br />
 |
| Key Performance Indicators (KPIs) are used to measure how close the actual test results are from the expectations. The
 |
| focus of KPIs vary depending on the type of testing performed: for performance testing, the KPIs will obviously be
 |
| interested in the time taken to execute the rules or tasks in the rule-set, for coverage, the focus will be on the set
 |
| of rule fired and the set of tasks executed, and for more business-oriented testing of correctness, the focus will be
 |
| on the accuracy of the response.<br />
 |
| </p>
 |
| <h5>
 |
| Correctness<br />
 |
| </h5>
 |
| <p>
 |
| For a decision-rich type of application, this is where most of the testing time and effort will be spent, using all
 |
| types of resources from developers, to business analysts, to business rule writers and subject matter experts.<br />
 |
| The goal here is to ensure that for a given request configuration, the system renders a response that conforms to the
 |
| company business goals and policies, contractual obligations and possibly to compliance rules imposed by external
 |
| agencies (laws and regulations). The impact of an incorrect decision can be profound, from lawsuits for non-compliance
 |
| to risky financial commitments.<br />
 |
| Testing the correctness of a business rules application necessarily involves the participation of SMEs, who will write
 |
| test cases and manually produce the expected response. This is a very time and resource consuming activity which should
 |
| be carefully taken into account when designing the test plan for an application.<br />
 |
| Alternatively, some test cases can be extracted from a legacy system database. The benefit here is obvious: the
 |
| database provides the number and the variety of test cases needed, and also guarantees the validity of the test
 |
| cases.<br />
 |
| </p>
 |
| <h5>
 |
| Regression<br />
 |
| </h5>
 |
| <p>
 |
| The goal of regression testing is to uncover regression problems, that is, problems which were not present in the
 |
| previous version of the application but appeared in the new version after a change was introduced. Regression testing
 |
| for rules is used to make sure that after fixing, or updating a rule service, the functionalities which were working
 |
| before are still working as expected. For example, after implementing a business change request which lowers the rates
 |
| in the state of California, regression testing should be applied to ensure that rates remained unchanged in all other
 |
| states.<br />
 |
| Besides verifying that correctness is conserved from one version to another, regression testing is also used to verify
 |
| that quality (for example, execution performance) is conserved.<br />
 |
| </p></mainDescription> |
| </org.eclipse.epf.uma:TaskDescription> |
