| <?php |
| /******************************************************************************* |
| * Copyright (c) 2015, 2016 Eclipse Foundation and others. |
| * All rights reserved. This program and the accompanying materials |
| * are made available under the terms of the Eclipse Public License v1.0 |
| * which accompanies this distribution, and is available at |
| * http://eclipse.org/legal/epl-v10.html |
| * |
| * Contributors: |
| * Eric Poirier (Eclipse Foundation) - Initial implementation |
| * Christopher Guindon (Eclipse Foundation) |
| *******************************************************************************/ |
| // This file must be included |
| if(basename(__FILE__) == basename($_SERVER['PHP_SELF'])){exit();} |
| ?> |
| |
| <h1 class="article-title"><?php echo $pageTitle; ?></h1> |
| |
| <p><i>Xpect is a framework that helps you to specify, test and discuss Xtext languages.</i></p> |
| <p> |
| <a target="_blank" href="http://www.xpect-tests.org">Xpect</a> has been around for a while <a target="_blank" href="https://github.com/meysholdt/Xpect">on GitHub</a>. The <a target="_blank" href="http://blog.moritz.eysholdt.de/2013/09/introduction-to-xpect.html">first announcement</a> is actually from 2013. Since then, even though there was never much marketing behind Xpect, it has seen a spectacular adoption among Xtext-projects. A large one, <a href="https://www.eclipse.org/n4js/">N4JS</a>, has recently moved to the Eclipse Foundation. |
| </p> |
| <p>Now we are happy to announce the next step: Xpect is <a target="_blank" href="https://twitter.com/i/web/status/893096871761195009">becoming an Eclipse project</a>. As of writing this, the <a href="https://projects.eclipse.org/reviews/xpect-creation-review">creation review is imminent</a>. |
| </p> |
| <p>Before I start explaining what Xpect is exactly, I would like to describe the context in which it is useful: Create Xtext |
| languages. Eclipse Xtext makes it very easy to implement smart editors, code generators, or compilers for your existing or self-designed |
| languages. The smart editors offers most of the nice features you probably love from Eclipse Java Tooling: Semantic Highlighting, |
| Content Assist, Live-Validation, Outline, Refactoring, etc. Also, there is Incremental Building, a global Index, and much more. Now when |
| you and your team implement an Xtext languages, you are in fact working on the grammar file, the ScopeProvider to define how cross |
| references are resolved, the validator, the generator, etc. During this work you typically:</p> |
| <ul> |
| <li>want to have automated tests. Tests give you this blissful peace of mind when refactoring your code and when rolling out the |
| next release.</li> |
| <li>review language syntax, semantics, and tooling behavior. Often, the review should be done by a domain expect who may not be a |
| Java programmer.</li> |
| <li>set up test stubs to make it easy for fellow developers to implement tests.</li> |
| </ul> |
| <p>Xpect supports these use cases by embedding tests and test-expectations inside comments inside documents of your language. |
| Basically, you create a an example document written in your Xtext-language and at any point you can put in a comment, saying, for |
| example, "here I expect a a validation error", "I expect the next cross reference to link to xyz". I will explain the actual syntax |
| later in this article. With this approach, Xpect is based on the following principles:</p> |
| <ul> |
| <li><b>Separate test implementation and test data+expectation</b>. This is great to have domain experts review your tests without the |
| needing to understand Java or Xtext's API.</li> |
| <li><b>Fast test creation</b>. Since a test is in fact a document written in your Xtext language, you can simply use the Xtext editor of |
| your language to create tests. Additionally, there is special editor that combines support for your language with |
| Xpect-syntax-support.</li> |
| <li><b>Tests should be self-explanatory on failures</b>. You, as a developer, break tests regularly when you refactor the underlying code. |
| Good tests really make a difference when a failing tests immediately lets you understand the cause of the problem. Tests that require |
| you to launch a debugger to unveil the problem are a time-sink.</li> |
| <li><b>Tests should run fast</b>. Almost all tests can run without OSGi/SWT and thus launch and finish in milliseconds. Furthermore, they |
| may use a shared setup. This allows tests to run tremendously fast and developers are actually running them locally, because that's |
| faster than triggering a Jenkins job.</li> |
| <li><b>It's OK when test expectations are a bit verbose</b>. This helps understanding them and makes good documentation. If you're worried about having slightly redundant tests that break more often than necessary: That's not a problem since the tests are easy to fix with |
| the "Synchronize Expectation and Actual Test Result" mechanism. Of course, this not an absolute statement and a good sense for "the |
| right amount" is needed.</li> |
| <li><b>Self-contained tests</b>. Self-contained tests are easier to understand tests: Both on failure and for domain experts, because no |
| exploration in other files or methods is needed to gather all relevant information.</li> |
| <li><b>No premature test failures</b>. Asserting a test result should happen with a single Assert-Statement, if possible. Unit tests in |
| which a failing assertion prevents subsequent assertions from being executed can be unnecessarily hard to analyze.</li> |
| </ul> |
| |
| <h2>One File, Two Languages</h2> |
| |
| <p align="center"><a target="_blank" href="images/xpect_editor_junit_view.gif"><img class="img-responsive" src="images/xpect_editor_junit_view.gif" /></a></p> |
| |
| <p>What you can see in the screen-cast above is a JUnit test that has been executed and which passed. The test class is <i>org.domainmodel.tests.validation.DMValidationTest</i> |
| and it has executed a file named <i>test1.dmodel.xt</i>. The file defines two test cases, one called <i>warnings</i> and one called <i>errors</i>. |
| On the right hand side you can see the file's contents in an editor. The editor generically combines support for two languages: First, |
| the language that is being tested. In the screenshot this is the Domainmodel language (*.dmodel) which ships with Xtext as an example |
| language. Secondly, the editor supports the Xpect language (*.xt). The two languages do not interfere with each other since the |
| Domainmodel language ignores text insides comments and the Xpect language ignores text that doesn't start with an "XPECT" keyword. The |
| editor applies a greenish background color to Xpect syntax. |
| </p> |
| |
| <p> |
| Let's take a closer look at the Xpect syntax. At the beginning of <i>test1.dmodel.xt</i>, there is a region called <span |
| style="font: 11.0px Monaco; background-color: #c5f9e0; color: #931a68">XPECT_SETUP</span>. It holds a reference to the JUnit test that |
| can run this file. Further down we find test cases such as: |
| </p> |
| |
| <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco; color: #4e9072">// capitalized property names are discouraged</p> |
| <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco"> |
| <span style="color: #4e9072;">// </span><span style="background-color: #c5f9e0; color: #931a68">XPECT</span> warnings --> <span |
| style="background-color: #c5f9e0;">"Name should start with a lowercase" at "Property1"</span> |
| </p> |
| <p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco">Property1 : String</p> |
| <p> |
| The first line is an optional title for the test. The <i>warnings</i> references is a JUnit test method (implemented in Java) and the |
| part following the <span style="white-space: nowrap;"><i>--></i></span> is the test expectation, which is passed as a parameter into |
| the JUnit test method. The test method can then compare the test expectation with what it calculated to be the actual test value and |
| pass or fail accordingly. For this example, the test expectation is composed of the error/warning message (<span |
| style="font: 11.0px Monaco; background-color: #c5f9e0;">Name should start with a lowercase</span>) and the text that would be |
| underlined by the red curly line in the editor (<span style="font: 11.0px Monaco; background-color: #c5f9e0;">Property1</span>). For |
| this validation test, the <span style="font: 11.0px Monaco; background-color: #c5f9e0; color: #931a68">XPECT</span> statement collects |
| all errors or warning occurring in the next line. A feature you might find very useful, is, that the <span |
| style="font: 11.0px Monaco; background-color: #c5f9e0; color: #931a68">XPECT</span> statement <i>consumes</i> error or warning |
| markers: An expected error or warning will <i>not</i> be shown as an error marker in the Eclipse editor. |
| </p> |
| |
| |
| <h2>Synchronize Expectations and Implementation</h2> |
| |
| <p>The fact that Xpect uses textual expectations and embeds them into DSL documents opens the door for another awesome (IMHO) feature: |
| Using the Eclipse comparison editor to inspect failed tests and to fix out-dated test expectations:</p> |
| |
| <p align="center"><a target="_blank" href="images/xpect_test_failure.gif"><img class="img-responsive" src="images/xpect_test_failure.gif" /></a></p> |
| |
| <p>When one or more tests fail and you want to fix them, it is crucial to quickly get an overview over all failed tests. With Xpect |
| you can not only see all failed test cases from one file in a single comparison editor, but there are also no assert statements which |
| sometimes prevent execution of follow-up assert statements and thereby hide valuable hints on why the test failed. The comparison |
| editor, as the name suggests, also lets you edit the test file.</p> |
| |
| <p> |
| With this approach you can still start test-first in the spirit of test-driven-development. However, later, when your implementation is |
| actually running, a second phase begins: Your implementation offers suggestion on how the test expectation should look like. It is your |
| job <i>review</i> this suggestion very carefully, and then, maybe, accept it as the better expectation. |
| </p> |
| |
| <h2>Reusable Test Library</h2> |
| <p> |
| In Xtext projects, there are several scenarios where it is reasonable to have test coverage. The validation test I explained earlier in |
| this article is just one of these scenarios. Xpect ships the Java-part for such tests <a target="_blank" |
| href="https://github.com/meysholdt/Xpect/tree/master/org.xpect.xtext.lib/src/org/xpect/xtext/lib/tests">as a library</a>. There is |
| also an <a target="_blank" |
| href="https://github.com/meysholdt/Xpect/tree/master/org.eclipse.xtext.example.domainmodel.xpect.tests/src/org/eclipse/xtext/example/domainmodel/xpect/tests">example |
| project</a> that demonstrates their usage. |
| </p> |
| <p>There are test for:</p> |
| <ul> |
| <li>The parser and Abstract Syntax Tree (AST) structure (demo only, no library).</li> |
| <li>Code generators implemented via Xtext's IGenerator interface.</li> |
| <li>Validation: Test for absence, presence, message and location of errors and warnings.</li> |
| <li>Linking: Verify a cross reference resolved to the intended model element</li> |
| <li>Scoping: Verify the expected names are included or excluded from a cross references's scope.</li> |
| <li>ResourceDescriptions: Verify a document exports the intended model elements with proper names.</li> |
| <li>JvmModelInferrer: For languages using Xbase, test the inferred JVM model</li> |
| </ul> |
| <p>There will be more tests in future versions of Xpect.</p> |
| |
| |
| <h2>Support for Standalone and Workspace Tests</h2> |
| <p>UI-independent parts of Xtext, such as the parser, can operate standalone (i.e. without OSGi and Eclipse Workspace). The same is |
| true for Xpect. For capabilities where Xtext does not require OSGi or an Eclipse Workspace, Xpect does not do so either. Consequently, |
| Xpect tests can be executed as plain JUnit test or Plug-In JUnit tests.</p> |
| |
| <p> |
| Since both scenarios require different kinds of setups, both can be configured separately in the <span |
| style="font: 11.0px Monaco; background-color: #c5f9e0; color: #931a68">XPECT_SETUP</span> section. When executed as plain JUnit test, |
| the ResourceSet-configuration is used and for Plug-In JUnit tests, the Workspace-Configuration is used. |
| </p> |
| |
| <p align="center"><a target="_blank" href="images/xpect_setup.png"><img class="img-responsive" src="images/xpect_setup_sm.png" /></a></p> |
| |
| <p>The screenshot also illustrates how an additional file can be loaded so that it is included in the current ResourceSet or Workspace |
| during test execution.</p> |
| |
| <h2>Suites: Combine Tests in the Same File</h2> |
| |
| <p>When explaining a specific concept of a language, it is helpful to look at the language concept from all sides: "This is the valid |
| syntax", "these scenarios are disallowed", "this is how cross references resolve", "this is how it is executed", etc. So far, testers |
| were required to create a new test file for every technical aspect. With Xpect test suites, however, it is possible to combine the |
| Java-parts of various tests into a single test suite.</p> |
| |
| <p align="center"><a href="images/xpect_suite.png"><img class="img-responsive" src="images/xpect_suite_sm.png" /></a></p> |
| |
| <p>The test suite <i>XtextTests</i> combines several tests so that in <i>CombiningMultipleTests.dmodel.xt</i> test methods from all these |
| tests can be used. This allows to group tests by language concept. |
| </p> |
| |
| <h2>Works with CI Builds</h2> |
| |
| <p>Xpect tests have proven to run fine with Gradle, Maven, Tycho, Surefire, and Jenkins. No special integration is necessary since |
| Xpect tests run as JUnit tests.</p> |
| |
| <h2>Source & Download</h2> |
| <p> You can find a link to the update site at <a target="_blank" href="http://www.xpect-tests.org/">www.xpect-tests.org</a>. The source code and an issue tracker are on the <a target="_blank" href="https://github.com/meysholdt/Xpect">GitHub project page</a>. |
| </p> |
| |
| <div class="bottomitem"> |
| <h3>About the Author</h3> |
| <div class="row"> |
| <div class="col-sm-12"> |
| <div class="row"> |
| <div class="col-sm-8"> |
| <img class="img-responsive" |
| src="/community/eclipse_newsletter/2017/august/images/moritz.jpeg" |
| alt="Moritz Eysholdt" /> |
| </div> |
| <div class="col-sm-16"> |
| <p class="author-name"> |
| Moritz Eysholdt<br /> |
| <a target="_blank" href="http://typefox.io/">TypeFox</a> |
| </p> |
| <ul class="author-link list-inline"> |
| <li><a class="btn btn-small btn-warning" target="_blank" href="https://twitter.com/docfx">Twitter</a></li> |
| <li><a class="btn btn-small btn-warning" target="_blank" href="https://github.com/meysholdt">GitHub</a></li> |
| <?php // echo $og; ?> |
| </ul> |
| </div> |
| </div> |
| </div> |
| </div> |
| </div> |
| |