Google Git
Sign in
eclipse / www.eclipse.org / rcptt / d51f50facf407f9fa8d07734a518428cd0134a59 / . / documentation / userguide / verifications / index.html
blob: 3340109ddf9209926a819c8a2cf1b026812c74de [file] [log] [blame]
<!DOCTYPE html>
<head>
<title>RCPTT Documentation Center</title>
<meta charset="UTF-8">
<link rel="stylesheet" href="https://www.eclipse.org/rcptt/shared/css/bootstrap.min.css">
<link rel="stylesheet" href="https://www.eclipse.org/rcptt/shared/css/main.css">
<link rel="stylesheet" href="https://www.eclipse.org/rcptt/shared/css/prism.css">
<link rel="icon" href="https://www.eclipse.org/rcptt/shared/img/favicon.ico">
<script src="https://www.eclipse.org/rcptt/shared/js/jquery.min.js"></script>
<script src="https://www.eclipse.org/rcptt/shared/js/bootstrap.min.js"></script>
<script src="https://www.eclipse.org/rcptt/shared/js/prism.js"></script>
<script src="https://www.eclipse.org/rcptt/shared/js/ecl.js"></script>
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-39589807-5', 'eclipse.org');
ga('send', 'pageview');
</script>
</head>
<body>
<div id="before-footer">
<header>
<nav class="navbar navbar-default navbar-static-top" role="navigation">
<div class="container">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1">
<span class="sr-only">Toggle navigation</span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<span class="navbar-brand">RCPTT Documentation Center</span>
</div>
<div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1">
<ul class="nav navbar-nav">
<li class="active" >
<a href="https://www.eclipse.org/rcptt/documentation/userguide/getstarted">User Guide</a>
</li>
<li >
<a href="https://www.eclipse.org/rcptt/documentation/faq/">FAQ</a>
</li>
<li>
<a href="http://eclipse.org/rcptt">Back to RCPTT Main Site</a>
</li>
<li>
<a href="http://eclipse.org">Eclipse.org</a>
</li>
</ul>
</div>
</nav>
</header>
<section class="content">
<div class="container">
<div class="row">
<div class="col-sm-3">
<div class="rcptt-sidebar">
<ul class="nav">
<li ><a href="https://www.eclipse.org/rcptt/documentation/userguide/getstarted/">Get Started</a>
</li>
<li ><a href="https://www.eclipse.org/rcptt/documentation/userguide/testeditor/">Test Case Editor</a>
</li>
<li ><a href="https://www.eclipse.org/rcptt/documentation/userguide/controlpanel">Control Panel</a>
</li>
<li ><a href="https://www.eclipse.org/rcptt/documentation/userguide/assertions/">Assertions</a>
</li>
<li ><a href="https://www.eclipse.org/rcptt/documentation/userguide/ecl/">Eclipse Command Language</a>
<ul class="nav">
<li >
<a href="https://www.eclipse.org/rcptt/documentation/userguide/ecl/new-command-guide/">New ECL Command Guide</a>
</li>
<li >
<a href="https://www.eclipse.org/rcptt/documentation/userguide/ecl/excel-import-export/">ECL commands for Excel import and export</a>
</li>
<li >
<a href="https://www.eclipse.org/rcptt/documentation/userguide/ecl/xml-import-export/">ECL commands for XML import and export</a>
</li>
</ul>
</li>
<li ><a href="https://www.eclipse.org/rcptt/documentation/userguide/contexts/">Contexts</a>
<ul class="nav">
<li >
<a href="https://www.eclipse.org/rcptt/documentation/userguide/contexts/workspace/">Workspace</a>
</li>
<li >
<a href="https://www.eclipse.org/rcptt/documentation/userguide/contexts/workbench/">Workbench</a>
</li>
<li >
<a href="https://www.eclipse.org/rcptt/documentation/userguide/contexts/launch/">Launch</a>
</li>
<li >
<a href="https://www.eclipse.org/rcptt/documentation/userguide/contexts/preferences/">Preferences</a>
</li>
<li >
<a href="https://www.eclipse.org/rcptt/documentation/userguide/contexts/parameters/">Parameters</a>
</li>
<li >
<a href="https://www.eclipse.org/rcptt/documentation/userguide/contexts/group/">Group</a>
</li>
<li >
<a href="https://www.eclipse.org/rcptt/documentation/userguide/contexts/folder/">Folder</a>
</li>
<li >
<a href="https://www.eclipse.org/rcptt/documentation/userguide/contexts/default/">Default</a>
</li>
<li >
<a href="https://www.eclipse.org/rcptt/documentation/userguide/contexts/super/">Super</a>
</li>
<li >
<a href="https://www.eclipse.org/rcptt/documentation/userguide/contexts/ecl/">ECL Script</a>
</li>
</ul>
</li>
<li class="active" ><a href="https://www.eclipse.org/rcptt/documentation/userguide/verifications/">Verifications</a>
<ul class="nav">
<li >
<a href="https://www.eclipse.org/rcptt/documentation/userguide/verifications/text/">Text</a>
</li>
<li >
<a href="https://www.eclipse.org/rcptt/documentation/userguide/verifications/treetable/">Table/Tree</a>
</li>
<li >
<a href="https://www.eclipse.org/rcptt/documentation/userguide/verifications/errorlog/">Error Log</a>
</li>
<li >
<a href="https://www.eclipse.org/rcptt/documentation/userguide/verifications/time/">Time</a>
</li>
</ul>
</li>
<li ><a href="https://www.eclipse.org/rcptt/documentation/userguide/procedures/">Procedures and Variables</a>
</li>
<li ><a href="https://www.eclipse.org/rcptt/documentation/userguide/runner/">Test Runner</a>
<ul class="nav">
<li >
<a href="https://www.eclipse.org/rcptt/documentation/userguide/runner/arguments/">Arguments</a>
</li>
</ul>
</li>
<li ><a href="https://www.eclipse.org/rcptt/documentation/userguide/maven/">Maven plugin</a>
</li>
<li ><a href="https://www.eclipse.org/rcptt/documentation/userguide/rap/">RAP</a>
<ul class="nav">
<li >
<a href="https://www.eclipse.org/rcptt/documentation/userguide/rap/install/">Install instructions</a>
</li>
<li >
<a href="https://www.eclipse.org/rcptt/documentation/userguide/rap/recordReplay/">Record and Replay</a>
</li>
<li >
<a href="https://www.eclipse.org/rcptt/documentation/userguide/rap/capability/">Capability Contexts</a>
</li>
<li >
<a href="https://www.eclipse.org/rcptt/documentation/userguide/rap/rap-commands/">ECL commands</a>
</li>
<li >
<a href="https://www.eclipse.org/rcptt/documentation/userguide/rap/rap-maven/">Maven plugin</a>
</li>
</ul>
</li>
<li ><a href="https://www.eclipse.org/rcptt/documentation/userguide/integration/test-rail-integration/">Integration with TestRail</a>
</li>
</ul>
</div>
</div>
<div class="col-sm-9">
<h1>Verifications</h1>
<p></p>
<p>A good functional test can be represented like this:</p>
<div class="screenshot">
<img src="https://www.eclipse.org/rcptt/shared/img/screenshot-verification-scheme-1.png"></img>
</div>
<p>Therefore, a test case takes an system under test in some well-defined state, performs some actions, which transfer a system into another state, and
then verifies that the final state is correct.</p>
<p>Thus, in a test case we need to specify the following:</p>
<ul>
<li>Initial state</li>
<li>Actions to perform</li>
<li>Expected final state</li>
</ul>
<p>Since the very beginning RCPTT had support for declarative description of an initial application state via contexts, and imperative
scripting language describing user actions, but there were no efficient way to verify a final state of an application - assertion
statements in scipt look very bulky and not very readable, they cannot be reused and it might take way too long to add assertions
for all aspects we want to check.</p>
<p>So we are filling this missing gap and introducing verifications - declarative reusable descriptions of various aspects
of an application state, and finally RCPTT test case description perfectly fits a picture from a beginning of this post:</p>
<div class="screenshot">
<img src="https://www.eclipse.org/rcptt/shared/img/screenshot-verification-scheme-2.png"></img>
</div>
<p>During setup of inital state, it does not make sense to describe it completely - we need to describe and ensure only parts of a state which are
relevant to our actions. For instance, if we are testing About dialog, we do not care about projects in workspace or open views, and if we are
testing Package Explorer view, we don't care about Java compiler settings.</p>
<p>Same with a final state - it makes sense to check only those aspects of an application state, which supposed to be affected by performed
actions (or those aspects which should not be affected, but in theory might be affected because of bugs).</p>
<p>Though both contexts and verifications are about a state, there is a couple of important differences between them:</p>
<ul>
<li>When context fails (i.e. we could not bring an application workspace into expected state), execution is immediately interrupted.
However if a verification fails, it makes sense to run other verifications too. Of course a test case is considered failing even if one
verification fails, but running all of them allows to collect more information about what is broken and what is working.</li>
<li>Context always specify an 'absolute' state, but during verification, for some aspects it matters to operate on a state delta -
the difference between initial and final states.</li>
</ul>
<p>Currently released version contains four kinds of verifications:</p>
<ul>
<li><b>Text verification</b>. Verifies that given widget contains expected text with (optionally) expected styles.
Thus, it is extremely easy to verify that source code is highlighted correctly, or verify that newly created Java class has correct contents.</li>
<li><b>Execution time verification</b>. Makes sure that a test case executes no longer than specified amount of time. This verification can be considered
as a constraint on a state change of a system clock.</li>
<li><b>Error Log verification</b>. Allows to verify the Error Log changes during a test case execution. Verification supports include/exclude rules,
error types, regular expressions, etc.</li>
<li><b>Tree/Table verification</b>. Allows you to check a state of a tree/table including it's children, icons, etc.</li>
</ul>
</div>
</div>
</div>
</section>
<script src="https://www.eclipse.org/rcptt/shared/js/main.js"></script>
</body>