XP/xp/guidances/guidelines/test_ideas_for_statechart_and_flow_diagrams.xmi - epf/org.eclipse.epf.libraries - Git at Google

 <?xml version="1.0" encoding="UTF-8"?>
 <org.eclipse.epf.uma:ContentDescription xmi:version="2.0" xmlns:xmi="http://www.omg.org/XMI" xmlns:org.eclipse.epf.uma="http://www.eclipse.org/epf/uma/1.0.3/uma.ecore" epf:version="1.0.0" xmi:id="-60dp5lxFJEpUarhchGCnHw" name="test_ideas_for_statechart_and_flow_diagrams,1.0347051690476123E-305" guid="-60dp5lxFJEpUarhchGCnHw" version="1.0.0">
   <mainDescription>&lt;a id=&quot;XE_state_machine__test_ideas_for&quot; name=&quot;XE_state_machine__test_ideas_for&quot;&gt;&lt;/a&gt;&lt;a
 id=&quot;XE_test_idea__for_state_machine&quot; name=&quot;XE_test_idea__for_state_machine&quot;&gt;&lt;/a&gt;
 &lt;h3&gt;
     &lt;a id=&quot;Introduction&quot; name=&quot;Introduction&quot;&gt;Introduction&lt;/a&gt;
 &lt;/h3&gt;
 &lt;p&gt;
     This guideline shows how to identify &lt;a class=&quot;elementLinkWithUserText&quot;
     href=&quot;./../../../xp/guidances/concepts/test-ideas_list,8.834380241450745E-306.html#TestIdeas&quot;
     guid=&quot;8.834380241450745E-306&quot;&gt;test ideas&lt;/a&gt; from statecharts and other design structures that consist mainly of nodes
     connected by arcs and that show something of the possible control flows of a program. The main goal of this testing is
     to traverse every arc in some test. If you've never exercised an arc, why do you think it will work when a customer
     does?
 &lt;/p&gt;
 &lt;h3&gt;
     &lt;a id=&quot;Implementation&quot; name=&quot;Implementation&quot;&gt;Testing the Implementation&lt;/a&gt;
 &lt;/h3&gt;
 &lt;p&gt;
     Consider this statechart:
 &lt;/p&gt;
 &lt;p align=&quot;center&quot;&gt;
     &lt;img height=&quot;253&quot; alt=&quot;&quot; src=&quot;resources/tstfrsdsg-img3.gif&quot; width=&quot;567&quot; /&gt;
 &lt;/p&gt;
 &lt;p class=&quot;picturetext&quot;&gt;
     Fig1: HVAC Statechart
 &lt;/p&gt;
 &lt;p&gt;
     Here's a first list of test ideas:
 &lt;/p&gt;
 &lt;ul&gt;
     &lt;li&gt;
         Idle state receives Too Hot event
     &lt;/li&gt;
     &lt;li&gt;
         Idle state receives Too Cool event
     &lt;/li&gt;
     &lt;li&gt;
         Cooling/Startup state receives Compressor Running event
     &lt;/li&gt;
     &lt;li&gt;
         Cooling/Ready state receives Fan Running event
     &lt;/li&gt;
     &lt;li&gt;
         Cooling/Running state receives OK event
     &lt;/li&gt;
     &lt;li&gt;
         Cooling/Running state receives Failure event
     &lt;/li&gt;
     &lt;li&gt;
         Failure state receives Failure Cleared event
     &lt;/li&gt;
     &lt;li&gt;
         Heating state receives OK event
     &lt;/li&gt;
     &lt;li&gt;
         Heating state receives Failure event
     &lt;/li&gt;
 &lt;/ul&gt;
 &lt;p&gt;
     These test ideas could all be exercised in a single test, or you could create several tests that each exercise a few.
     As with all test design, strive for a balance between the ease of implementation of many simple tests and the
     additional defect-finding power of complex tests. (See &lt;a class=&quot;elementLinkWithUserText&quot;
     href=&quot;./../../../xp/guidances/concepts/test-ideas_list,8.834380241450745E-306.html#TestDesignUsingTheList&quot;
     guid=&quot;8.834380241450745E-306&quot;&gt;&quot;test design using the list&quot;&lt;/a&gt; in the &lt;a class=&quot;elementLinkWithUserText&quot;
     href=&quot;./../../../xp/guidances/concepts/test-ideas_list,8.834380241450745E-306.html&quot;
     guid=&quot;8.834380241450745E-306&quot;&gt;Concept: Test Ideas List&lt;/a&gt; page.) If you have use case scenarios that describe certain
     paths through the statechart, you should favor tests that take those paths.
 &lt;/p&gt;
 &lt;p&gt;
     In any case, the tests should check that all actions required by the statechart actually take place. For example, is
     the alarm started on entry to the Failure state, then stopped upon exit?
 &lt;/p&gt;
 &lt;p&gt;
     The test should also check that the transition leads to the correct next state. That can be a difficult problem when
     the states are invisible from the outside. The only way to detect an incorrect state is to inject some sequence of
     events that leads to incorrect output. More precisely, you would need to construct a follow-on sequence of events whose
     externally-visible results for the &lt;i&gt;correct&lt;/i&gt; state differ from those that the same sequence would provoke from
     each possible &lt;i&gt;incorrect&lt;/i&gt; state.
 &lt;/p&gt;
 &lt;p&gt;
     In the example above, how would you know that the Failure Cleared event in the Failure state correctly led to the Idle
     state, instead of staying in the Failure state? You might trust that the stopping of the Alarm meant that transition
     had been made, but it might be better to check by lowering the temperature enough to make the heater start or raising
     it enough to turn on cooling. If something happens, you're more confident that the transition was correct. If nothing
     happens, it's likely the device stayed in the Failure state.
 &lt;/p&gt;
 &lt;p&gt;
     At the very least, determining whether the resulting state is correct complicates test design. It is often better to
     make the state machine explicit and make its states visible to the tests.
 &lt;/p&gt;
 &lt;h4&gt;
     Other statechart constructs
 &lt;/h4&gt;
 &lt;p&gt;
     Statecharts consist of more than arcs and arrows. Here is a list of statechart constructs and the effect they have on
     the test idea list.
 &lt;/p&gt;
 &lt;h5&gt;
     Event actions, entry actions, and exit actions
 &lt;/h5&gt;
 &lt;p&gt;
     These do not generate test ideas per se. Rather, the tests should check that the actions behave as specified. If the
     actions represent substantial programs, those programs must be tested. The test ideas for the programs might be
     combined with test ideas from the statechart, but it's probably more manageable to separate them. Make the decision
     based on the effort involved and on your suspicion that there might be interactions between events. That is, if a
     particular action on one arc cannot possibly share data with an action on another arc, there is no reason to exercise
     the two actions in the same test (as you would if they were part of the same path through a statechart test).
 &lt;/p&gt;
 &lt;h5&gt;
     Guard conditions
 &lt;/h5&gt;
 &lt;p&gt;
     Guard conditions are boolean expressions. The test ideas for guard conditions are derived as described in &lt;a
     class=&quot;elementLinkWithType&quot;
     href=&quot;./../../../xp/guidances/guidelines/test_ideas_for_booleans_and_boundaries,1.7150344523489172E-305.html&quot;
     guid=&quot;1.7150344523489172E-305&quot;&gt;Guideline: Test Ideas for Booleans and Boundaries&lt;/a&gt;.
 &lt;/p&gt;
 &lt;p&gt;
     In the example above, the Too Cool transition from the Idle state is guarded with [restart time &amp;gt;= 5 mins]. That
     leads to two separate test ideas:
 &lt;/p&gt;
 &lt;ul&gt;
     &lt;li&gt;
         Idle state receives Too Cool event when restart time is five minutes (transition taken)
     &lt;/li&gt;
     &lt;li&gt;
         Idle state receives Too Cool event when restart time is just less than five minutes (transition blocked)
     &lt;/li&gt;
 &lt;/ul&gt;
 &lt;p&gt;
     In both cases, any test that uses the test idea should check that the correct state is reached.
 &lt;/p&gt;
 &lt;h5&gt;
     Internal transitions
 &lt;/h5&gt;
 &lt;p&gt;
     An internal transition adds the same sort of ideas to a test idea list as an external transition does. It's merely that
     the next state is the same as the original state. It would be prudent to set up the test such that the state's entry
     and exit actions would cause an observable effect if they were incorrectly triggered.
 &lt;/p&gt;
 &lt;h5&gt;
     Nested states
 &lt;/h5&gt;
 &lt;p&gt;
     When constructing tests, set them up such that entry and exit events of the composite state have observable effects.
     You want to notice if they're skipped.
 &lt;/p&gt;
 &lt;h5&gt;
     Concurrent substates
 &lt;/h5&gt;
 &lt;p&gt;
     Testing of concurrency falls outside of the scope of developer testing.
 &lt;/p&gt;
 &lt;h5&gt;
     Deferred events
 &lt;/h5&gt;
 &lt;p&gt;
     If you suspect an event might be handled differently depending on whether it was deferred and queued rather than
     generated while the program was actually in the receiving state, you might test those two cases.
 &lt;/p&gt;
 &lt;p&gt;
     If the event in the receiving state has a guard condition, consider the ramifications of changes to the condition's
     variables between the time the event is generated and the time it is received.
 &lt;/p&gt;
 &lt;p&gt;
     If more than one state can handle a deferred event, consider testing deferral to each of the possible receiving states.
     Perhaps the implementation assumes that the &quot;obvious&quot; state will handle the event.
 &lt;/p&gt;
 &lt;h5&gt;
     History states
 &lt;/h5&gt;
 &lt;p&gt;
     Here is an example of a history state:
 &lt;/p&gt;
 &lt;p align=&quot;center&quot;&gt;
     &lt;img height=&quot;211&quot; alt=&quot;&quot; src=&quot;resources/md_state3.gif&quot; width=&quot;412&quot; /&gt;
 &lt;/p&gt;
 &lt;p class=&quot;picturetext&quot;&gt;
     Fig2: History State Example
 &lt;/p&gt;
 &lt;p&gt;
     The transition into the history state represents three real transitions, and thus three test ideas:
 &lt;/p&gt;
 &lt;ul&gt;
     &lt;li&gt;
         BackupUp event in Command state leads to Collecting state
     &lt;/li&gt;
     &lt;li&gt;
         BackupUp event in Command state leads to Copying state
     &lt;/li&gt;
     &lt;li&gt;
         BackupUp event in Command state leads to CleaningUp state
     &lt;/li&gt;
 &lt;/ul&gt;
 &lt;h5&gt;
     Chain states
 &lt;/h5&gt;
 &lt;p&gt;
     Chain states do not seem to have any implications for test design, except that they introduce more actions that need to
     be checked.
 &lt;/p&gt;
 &lt;h3&gt;
     &lt;a id=&quot;Design&quot; name=&quot;Design&quot;&gt;Testing the Design&lt;/a&gt;
 &lt;/h3&gt;
 &lt;p&gt;
     The preceding discussion focuses on checking whether the implementation matches the design. But the design might also
     be wrong. While examining the design to find test ideas, also check for two types of problems:
 &lt;/p&gt;
 &lt;p&gt;
     &lt;b&gt;Missing events.&lt;/b&gt; The statechart shows a state's response to events &lt;i&gt;that the designer anticipated could arrive
     in that state&lt;/i&gt;. It's not unknown for designers to overlook events. For example, in this statechart (repeated from
     the top of the page), perhaps the designer forgot that a failure can occur in the Ready substate of Cooling, not just
     when the fan is Running.
 &lt;/p&gt;
 &lt;p align=&quot;center&quot;&gt;
     &lt;img height=&quot;253&quot; alt=&quot;&quot; src=&quot;resources/tstfrsdsg-img3.gif&quot; width=&quot;567&quot; /&gt;
 &lt;/p&gt;
 &lt;p class=&quot;picturetext&quot;&gt;
     Fig3: HVAC Statechart
 &lt;/p&gt;
 &lt;p&gt;
     For this reason, it's wise to ask, for each state, whether any of the events that apply to other states might apply to
     this one. If you discover that one does, correct your design.
 &lt;/p&gt;
 &lt;p&gt;
     &lt;b&gt;Incomplete or missing guard conditions.&lt;/b&gt; Similarly, perhaps guard conditions on one transition will suggest guard
     conditions on others. For example, the above statechart takes care not to restart the heater too often, but there is no
     such restriction on the cooling system. Should there be?
 &lt;/p&gt;
 &lt;p&gt;
     It is also possible that variables used on one guard condition will suggest that other guard conditions are too simple.
 &lt;/p&gt;
 &lt;h3&gt;
     &lt;a id=&quot;Interactions&quot; name=&quot;Interactions&quot;&gt;Testing Interactions&lt;/a&gt;
 &lt;/h3&gt;
 &lt;p&gt;
     Testing each arc in a graph is by no means complete testing. For example, suppose the start state initializes a
     variable to 0, state Setter sets it to 5, and state Divider divides it into 100 (100/variable). If there's a path from
     the start state to Divider that does not pass through Setter, you have a divide-by-zero exception. If the statechart
     has many states, simply exercising each arc might miss that path.
 &lt;/p&gt;
 &lt;p&gt;
     Except for very simple statecharts, testing every path is infeasible. In practice, tests that are complex and
     correspond to use case scenarios are often sufficient. If you desire stronger tests, consider requiring a path from
     each state where a datum is given a value to each state that uses it.
 &lt;/p&gt;
 &lt;br /&gt;
 &lt;br /&gt;</mainDescription>
 </org.eclipse.epf.uma:ContentDescription>
	<?xml version="1.0" encoding="UTF-8"?>
	<org.eclipse.epf.uma:ContentDescription xmi:version="2.0" xmlns:xmi="http://www.omg.org/XMI" xmlns:org.eclipse.epf.uma="http://www.eclipse.org/epf/uma/1.0.3/uma.ecore" epf:version="1.0.0" xmi:id="-60dp5lxFJEpUarhchGCnHw" name="test_ideas_for_statechart_and_flow_diagrams,1.0347051690476123E-305" guid="-60dp5lxFJEpUarhchGCnHw" version="1.0.0">
	<mainDescription><a id="XE_state_machine__test_ideas_for" name="XE_state_machine__test_ideas_for"></a><a
	id="XE_test_idea__for_state_machine" name="XE_test_idea__for_state_machine"></a>
	<h3>
	<a id="Introduction" name="Introduction">Introduction</a>
	</h3>
	<p>
	This guideline shows how to identify <a class="elementLinkWithUserText"
	href="./../../../xp/guidances/concepts/test-ideas_list,8.834380241450745E-306.html#TestIdeas"
	guid="8.834380241450745E-306">test ideas</a> from statecharts and other design structures that consist mainly of nodes
	connected by arcs and that show something of the possible control flows of a program. The main goal of this testing is
	to traverse every arc in some test. If you've never exercised an arc, why do you think it will work when a customer
	does?
	</p>
	<h3>
	<a id="Implementation" name="Implementation">Testing the Implementation</a>
	</h3>
	<p>
	Consider this statechart:
	</p>
	<p align="center">
	<img height="253" alt="" src="resources/tstfrsdsg-img3.gif" width="567" />
	</p>
	<p class="picturetext">
	Fig1: HVAC Statechart
	</p>
	<p>
	Here's a first list of test ideas:
	</p>
	<ul>
	<li>
	Idle state receives Too Hot event
	</li>
	<li>
	Idle state receives Too Cool event
	</li>
	<li>
	Cooling/Startup state receives Compressor Running event
	</li>
	<li>
	Cooling/Ready state receives Fan Running event
	</li>
	<li>
	Cooling/Running state receives OK event
	</li>
	<li>
	Cooling/Running state receives Failure event
	</li>
	<li>
	Failure state receives Failure Cleared event
	</li>
	<li>
	Heating state receives OK event
	</li>
	<li>
	Heating state receives Failure event
	</li>
	</ul>
	<p>
	These test ideas could all be exercised in a single test, or you could create several tests that each exercise a few.
	As with all test design, strive for a balance between the ease of implementation of many simple tests and the
	additional defect-finding power of complex tests. (See <a class="elementLinkWithUserText"
	href="./../../../xp/guidances/concepts/test-ideas_list,8.834380241450745E-306.html#TestDesignUsingTheList"
	guid="8.834380241450745E-306">"test design using the list"</a> in the <a class="elementLinkWithUserText"
	href="./../../../xp/guidances/concepts/test-ideas_list,8.834380241450745E-306.html"
	guid="8.834380241450745E-306">Concept: Test Ideas List</a> page.) If you have use case scenarios that describe certain
	paths through the statechart, you should favor tests that take those paths.
	</p>
	<p>
	In any case, the tests should check that all actions required by the statechart actually take place. For example, is
	the alarm started on entry to the Failure state, then stopped upon exit?
	</p>
	<p>
	The test should also check that the transition leads to the correct next state. That can be a difficult problem when
	the states are invisible from the outside. The only way to detect an incorrect state is to inject some sequence of
	events that leads to incorrect output. More precisely, you would need to construct a follow-on sequence of events whose
	externally-visible results for the <i>correct</i> state differ from those that the same sequence would provoke from
	each possible <i>incorrect</i> state.
	</p>
	<p>
	In the example above, how would you know that the Failure Cleared event in the Failure state correctly led to the Idle
	state, instead of staying in the Failure state? You might trust that the stopping of the Alarm meant that transition
	had been made, but it might be better to check by lowering the temperature enough to make the heater start or raising
	it enough to turn on cooling. If something happens, you're more confident that the transition was correct. If nothing
	happens, it's likely the device stayed in the Failure state.
	</p>
	<p>
	At the very least, determining whether the resulting state is correct complicates test design. It is often better to
	make the state machine explicit and make its states visible to the tests.
	</p>
	<h4>
	Other statechart constructs
	</h4>
	<p>
	Statecharts consist of more than arcs and arrows. Here is a list of statechart constructs and the effect they have on
	the test idea list.
	</p>
	<h5>
	Event actions, entry actions, and exit actions
	</h5>
	<p>
	These do not generate test ideas per se. Rather, the tests should check that the actions behave as specified. If the
	actions represent substantial programs, those programs must be tested. The test ideas for the programs might be
	combined with test ideas from the statechart, but it's probably more manageable to separate them. Make the decision
	based on the effort involved and on your suspicion that there might be interactions between events. That is, if a
	particular action on one arc cannot possibly share data with an action on another arc, there is no reason to exercise
	the two actions in the same test (as you would if they were part of the same path through a statechart test).
	</p>
	<h5>
	Guard conditions
	</h5>
	<p>
	Guard conditions are boolean expressions. The test ideas for guard conditions are derived as described in <a
	class="elementLinkWithType"
	href="./../../../xp/guidances/guidelines/test_ideas_for_booleans_and_boundaries,1.7150344523489172E-305.html"
	guid="1.7150344523489172E-305">Guideline: Test Ideas for Booleans and Boundaries</a>.
	</p>
	<p>
	In the example above, the Too Cool transition from the Idle state is guarded with [restart time &gt;= 5 mins]. That
	leads to two separate test ideas:
	</p>
	<ul>
	<li>
	Idle state receives Too Cool event when restart time is five minutes (transition taken)
	</li>
	<li>
	Idle state receives Too Cool event when restart time is just less than five minutes (transition blocked)
	</li>
	</ul>
	<p>
	In both cases, any test that uses the test idea should check that the correct state is reached.
	</p>
	<h5>
	Internal transitions
	</h5>
	<p>
	An internal transition adds the same sort of ideas to a test idea list as an external transition does. It's merely that
	the next state is the same as the original state. It would be prudent to set up the test such that the state's entry
	and exit actions would cause an observable effect if they were incorrectly triggered.
	</p>
	<h5>
	Nested states
	</h5>
	<p>
	When constructing tests, set them up such that entry and exit events of the composite state have observable effects.
	You want to notice if they're skipped.
	</p>
	<h5>
	Concurrent substates
	</h5>
	<p>
	Testing of concurrency falls outside of the scope of developer testing.
	</p>
	<h5>
	Deferred events
	</h5>
	<p>
	If you suspect an event might be handled differently depending on whether it was deferred and queued rather than
	generated while the program was actually in the receiving state, you might test those two cases.
	</p>
	<p>
	If the event in the receiving state has a guard condition, consider the ramifications of changes to the condition's
	variables between the time the event is generated and the time it is received.
	</p>
	<p>
	If more than one state can handle a deferred event, consider testing deferral to each of the possible receiving states.
	Perhaps the implementation assumes that the "obvious" state will handle the event.
	</p>
	<h5>
	History states
	</h5>
	<p>
	Here is an example of a history state:
	</p>
	<p align="center">
	<img height="211" alt="" src="resources/md_state3.gif" width="412" />
	</p>
	<p class="picturetext">
	Fig2: History State Example
	</p>
	<p>
	The transition into the history state represents three real transitions, and thus three test ideas:
	</p>
	<ul>
	<li>
	BackupUp event in Command state leads to Collecting state
	</li>
	<li>
	BackupUp event in Command state leads to Copying state
	</li>
	<li>
	BackupUp event in Command state leads to CleaningUp state
	</li>
	</ul>
	<h5>
	Chain states
	</h5>
	<p>
	Chain states do not seem to have any implications for test design, except that they introduce more actions that need to
	be checked.
	</p>
	<h3>
	<a id="Design" name="Design">Testing the Design</a>
	</h3>
	<p>
	The preceding discussion focuses on checking whether the implementation matches the design. But the design might also
	be wrong. While examining the design to find test ideas, also check for two types of problems:
	</p>
	<p>
	<b>Missing events.</b> The statechart shows a state's response to events <i>that the designer anticipated could arrive
	in that state</i>. It's not unknown for designers to overlook events. For example, in this statechart (repeated from
	the top of the page), perhaps the designer forgot that a failure can occur in the Ready substate of Cooling, not just
	when the fan is Running.
	</p>
	<p align="center">
	<img height="253" alt="" src="resources/tstfrsdsg-img3.gif" width="567" />
	</p>
	<p class="picturetext">
	Fig3: HVAC Statechart
	</p>
	<p>
	For this reason, it's wise to ask, for each state, whether any of the events that apply to other states might apply to
	this one. If you discover that one does, correct your design.
	</p>
	<p>
	<b>Incomplete or missing guard conditions.</b> Similarly, perhaps guard conditions on one transition will suggest guard
	conditions on others. For example, the above statechart takes care not to restart the heater too often, but there is no
	such restriction on the cooling system. Should there be?
	</p>
	<p>
	It is also possible that variables used on one guard condition will suggest that other guard conditions are too simple.
	</p>
	<h3>
	<a id="Interactions" name="Interactions">Testing Interactions</a>
	</h3>
	<p>
	Testing each arc in a graph is by no means complete testing. For example, suppose the start state initializes a
	variable to 0, state Setter sets it to 5, and state Divider divides it into 100 (100/variable). If there's a path from
	the start state to Divider that does not pass through Setter, you have a divide-by-zero exception. If the statechart
	has many states, simply exercising each arc might miss that path.
	</p>
	<p>
	Except for very simple statecharts, testing every path is infeasible. In practice, tests that are complex and
	correspond to use case scenarios are often sufficient. If you desire stronger tests, consider requiring a path from
	each state where a datum is given a value to each state that uses it.
	</p>
	<br />
	<br /></mainDescription>
	</org.eclipse.epf.uma:ContentDescription>