| <?xml version="1.0" encoding="UTF-8"?> |
| <?asciidoc-toc maxdepth="5"?> |
| <?asciidoc-numbered maxdepth="5"?> |
| <book xmlns="http://docbook.org/ns/docbook" xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en"> |
| <info> |
| <title>N4JS Design Specification</title> |
| <date>2019-08-07</date> |
| <author> |
| <personname> |
| <firstname>2019-08-07 15:02:40 CEST</firstname> |
| </personname> |
| </author> |
| <authorinitials>{</authorinitials> |
| <style> |
| .admonitionblock td.icon .icon-todo:before{content:"\f249";color:#f4ee42} |
| </style> |
| </info> |
| <preface> |
| <title></title> |
| <simpara role="center"><emphasis role="strong">Last Updated: 2019-08-07</emphasis></simpara> |
| <simpara role="center"><emphasis role="strong">Authors:</emphasis><?asciidoc-br?> |
| Jens von Pilgrim, Jakub Siberski, Mark-Oliver Reiser, Torsten Krämer, Ákos Kitta, Sebastian Zarnekow, Lorenzo Bettini, Jörg Reichert, Kristian Duske, Marcus Mews, Minh Quang Tran, Luca Beurer-Kellner</simpara> |
| <simpara><?asciidoc-pagebreak?></simpara> |
| <simpara>This document contains the N4JS Design and Implementation documentation.</simpara> |
| <simpara><?asciidoc-pagebreak?></simpara> |
| </preface> |
| <chapter xml:id="_introduction"> |
| <title>Introduction</title> |
| <simpara>This document describes design aspects of the N4JS compiler and IDE. It relies on the following N4JS related specifications:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>N4JS Language Specification [<link linkend="N4JSSpec">N4JSSpec</link>]</simpara> |
| </listitem> |
| </itemizedlist> |
| <section xml:id="notation"> |
| <title>Notation</title> |
| <simpara>We reuse the notation specified in [<link linkend="N4JSSpec">N4JSSpec</link>].</simpara> |
| </section> |
| <section xml:id="sec:IDE_Overview"> |
| <title>IDE Components</title> |
| <simpara>The N4JS and N4JSIDE components are organized via features. The following features with included plugins are defined |
| (the common prefix "org.eclipse.n4js" is omitted at the plugin name):</simpara> |
| <informaltable frame="all" rowsep="1" colsep="1"> |
| <tgroup cols="3"> |
| <colspec colname="col_1" colwidth="5.8823*"/> |
| <colspec colname="col_2" colwidth="11.7647*"/> |
| <colspec colname="col_3" colwidth="82.353*"/> |
| <thead> |
| <row> |
| <entry align="left" valign="top">Feature</entry> |
| <entry align="left" valign="top">Plugin</entry> |
| <entry align="left" valign="top">Description</entry> |
| </row> |
| </thead> |
| <tbody> |
| <row> |
| <entry align="left" valign="top" namest="col_1" nameend="col_2"><simpara><emphasis role="strong">org.eclipse.n4js.lang.sdk</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara>N4JS core language with parser, validation etc.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>org.eclipse.n4js</simpara></entry> |
| <entry align="left" valign="top"><simpara>Xtext grammar with generator and custom code for N4JS, scoping (and binding) implementation, basic validation (and Xsemantics type system).</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>doc</simpara></entry> |
| <entry align="left" valign="top"><simpara>(in doc folder) General documentation (including web page) written in AsciiDoc</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>external.libraries</simpara></entry> |
| <entry align="left" valign="top"><simpara>Support for N4JS libraries shipped with the IDE, i.e. core N4JS library and mangelhaft.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>ui</simpara></entry> |
| <entry align="left" valign="top"><simpara>UI components for N4JS, e.g., proposal provider, labels, outline, quickfixes.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>jsdoc</simpara></entry> |
| <entry align="left" valign="top"><simpara>Parser and model for JSDoc</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>external.libraries.update</simpara></entry> |
| <entry align="left" valign="top"><simpara><emphasis role="strong">Not included in feature</emphasis>. Updates the external library plugin</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top" namest="col_1" nameend="col_2"><simpara><emphasis role="strong">org.eclipse.n4js.ts.sdk</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara>Type System</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>ts</simpara></entry> |
| <entry align="left" valign="top"><simpara>Xtext grammar with generator and custom code for type expressions and standalone type definitions.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>ts.model</simpara></entry> |
| <entry align="left" valign="top"><simpara>Xcore based types model with helper classes etc.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>ts.ui</simpara></entry> |
| <entry align="left" valign="top"><simpara>Xtext generated UI for type system, not really used as this TS files are not editable by users.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top" namest="col_1" nameend="col_2"><simpara><emphasis role="strong">org.eclipse.n4js.unicode.sdk</emphasis></simpara></entry> |
| <entry align="left" valign="top"></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>common.unicode</simpara></entry> |
| <entry align="left" valign="top"><simpara>Xtext grammar with generator and custom code used by all other grammars for proper unicode support.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top" namest="col_1" nameend="col_2"><simpara><emphasis role="strong">org.eclipse.n4js.regex.sdk</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara>Regular expression grammar and UI, used by N4JS grammar and UI</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>regex</simpara></entry> |
| <entry align="left" valign="top"><simpara>Xtext grammar with generator and custom code used by N4JS grammars for regular expressions.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>regex.ui</simpara></entry> |
| <entry align="left" valign="top"><simpara>UI components for regular expressions, e.g., proposal provider, labels, outline, quickfixes.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top" namest="col_1" nameend="col_2"><simpara><emphasis role="strong">org.eclipse.n4js.sdk</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara>This feature defines the N4JSIDE. It contains core UI plugins and all includes (almost all) other features!</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>environments</simpara></entry> |
| <entry align="left" valign="top"><simpara>Utility plugin, registers n4scheme for EMF proxy resolution.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>model</simpara></entry> |
| <entry align="left" valign="top"><simpara>Xcore based N4JS model with helper classes etc.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>product</simpara></entry> |
| <entry align="left" valign="top"><simpara>N4JSIDE main application.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>releng.utils</simpara></entry> |
| <entry align="left" valign="top"><simpara>(in releng folder) Contains utility classes only used for building the system, e.g., tools for generating antlr based parser with extended features.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>utils</simpara></entry> |
| <entry align="left" valign="top"><simpara>general utilities</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>utils.ui</simpara></entry> |
| <entry align="left" valign="top"><simpara>general UI utilities</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top" namest="col_1" nameend="col_2"><simpara><emphasis role="strong">org.eclipse.n4js.compiler.sdk</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara>Compilers and Transpilers</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>generator.common</simpara></entry> |
| <entry align="left" valign="top"><simpara><emphasis role="strong">Not included in feature, logically associated.</emphasis></simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>generator.headless</simpara></entry> |
| <entry align="left" valign="top"><simpara>N4JS headless generator (i.e. command line compiler).</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>transpiler</simpara></entry> |
| <entry align="left" valign="top"><simpara>Generic transpiler infrastructure</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>transpiler.es</simpara></entry> |
| <entry align="left" valign="top"><simpara>Transpiler to compile to EcmaScript</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top" namest="col_1" nameend="col_2"><simpara><emphasis role="strong">org.eclipse.n4js.json.sdk</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara>N4JS JSON</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>json</simpara></entry> |
| <entry align="left" valign="top"><simpara>Xtext grammar with generator and custom code for a extensible JSON language support. Used in N4JS for the project description in terms of a <literal>package.json</literal> file.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>json.ui</simpara></entry> |
| <entry align="left" valign="top"><simpara>UI components for extensible JSON language support, e.g., proposal provider, labels, outline.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>json.model</simpara></entry> |
| <entry align="left" valign="top"><simpara><emphasis role="strong">Not included in feature, logically associated.</emphasis> Xcore based model for the JSON language.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top" namest="col_1" nameend="col_2"><simpara><emphasis role="strong">org.eclipse.n4js.semver.sdk</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara>Semantic version string support.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>semver</simpara></entry> |
| <entry align="left" valign="top"><simpara>Parser and tools for semantic version strings.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>semver.ui</simpara></entry> |
| <entry align="left" valign="top"><simpara>UI tools for semantic version strings.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>semver.model</simpara></entry> |
| <entry align="left" valign="top"><simpara><emphasis role="strong">Not included in feature, logically associated.</emphasis> Xcore model of semantic version strings.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top" namest="col_1" nameend="col_2"><simpara><emphasis role="strong">org.eclipse.n4js.runner.sdk</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara>Runners for executing N4JS or JavaScript code</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>runner</simpara></entry> |
| <entry align="left" valign="top"><simpara>Generic interfaces and helper for runners, i.e. JavaScript engines executing N4JS or JavaScript code.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>runner.chrome</simpara></entry> |
| <entry align="left" valign="top"><simpara>Runner for executing N4JS or JavaScript with Chrome.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>runner.chrome.ui</simpara></entry> |
| <entry align="left" valign="top"><simpara>UI classes for launching the Chrome runner via the org.eclipse.debug.ui</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>runner.nodejs</simpara></entry> |
| <entry align="left" valign="top"><simpara>Runner for executing N4JS or JavaScript with node.js.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>runner.nodejs.ui</simpara></entry> |
| <entry align="left" valign="top"><simpara>UI classes for launching the node.js runner via the org.eclipse.debug.ui</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>runner.ui</simpara></entry> |
| <entry align="left" valign="top"><simpara>Generic interfaces for configuring N4JS runner via the debug ui.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top" namest="col_1" nameend="col_2"><simpara><emphasis role="strong">org.eclipse.n4js.tester.sdk</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara>Runners and UI for tests (via mangelhaft).</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>tester</simpara></entry> |
| <entry align="left" valign="top"><simpara>Generic interfaces and helper for testers, i.e. JavaScript engines executing N4JS tests (using mangelhaft).</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>tester.nodejs</simpara></entry> |
| <entry align="left" valign="top"><simpara>Tester based on the nodejs runner for executing mangelhaft tests with node.js</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>tester.nodejs.ui</simpara></entry> |
| <entry align="left" valign="top"><simpara>UI for showing test results.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>tester.ui</simpara></entry> |
| <entry align="left" valign="top"><simpara>Configuration of tests via the debug UI.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top" namest="col_1" nameend="col_2"><simpara><emphasis role="strong">org.eclipse.n4js.jsdoc2spec.sdk</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara>JSDoc 2 Specification</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>jsdoc2spec</simpara></entry> |
| <entry align="left" valign="top"><simpara>Exporter to generate API documentation with specification tests awareness</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>jsdoc2spec.ui</simpara></entry> |
| <entry align="left" valign="top"><simpara>UI for API doc exporter</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top" namest="col_1" nameend="col_2"><simpara><emphasis role="strong">org.eclipse.n4js.xpect.sdk</emphasis></simpara></entry> |
| <entry align="left" valign="top"></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>xpect</simpara></entry> |
| <entry align="left" valign="top"><simpara>Xpect test methods.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>xpect.ui</simpara></entry> |
| <entry align="left" valign="top"><simpara>UI for running Xpext tests methods from the N4JSIDE (for creating bug reports).</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top" namest="col_1" nameend="col_2"><simpara><emphasis role="strong">org.eclipse.n4js.smith.sdk</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara>Feature for internal N4JS IDE plugins only intended for development (for example, the AST Graph view).</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>smith</simpara></entry> |
| <entry align="left" valign="top"><simpara>Non-UI classes for tools for smiths, that is, tools for developers of the N4JS IDE such as AST views etc.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>smith.ui</simpara></entry> |
| <entry align="left" valign="top"><simpara>UI classes for tools for smiths, that is, tools for developers of the N4JS IDE such as AST views etc.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top" namest="col_1" nameend="col_2"><simpara><emphasis role="strong">org.eclipse.n4js.tests.helper.sdk</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara>Test helpers.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top" namest="col_1" nameend="col_2"><simpara><emphasis role="strong">org.eclipse.n4js.dependencies.sdk</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara>Collection of all external non-ui dependencies, used for local mirroring of update sites.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top" namest="col_1" nameend="col_2"><simpara><emphasis role="strong">org.eclipse.n4js.dependencies.ui.sdk</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara>Collection of all external ui dependencies, used for local mirroring of update sites.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top" namest="col_1" nameend="col_3"><simpara><emphasis role="strong">uncategorized plugins</emphasis></simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>flowgraphs</simpara></entry> |
| <entry align="left" valign="top"><simpara>Control and data flow graph model and computer.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top" namest="col_1" nameend="col_2"><simpara><emphasis role="strong">Fragments</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara>not associated to features, only listed here for completeness</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"></entry> |
| <entry align="left" valign="top"><simpara>utils.logging</simpara></entry> |
| <entry align="left" valign="top"><simpara>Fragment only, configuration for loggers, in particular for the product and for the tests</simpara></entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| <section xml:id="sec:Naming_Conventions"> |
| <title>Naming Conventions</title> |
| <simpara>In the above sections, tests were omitted. We use the following naming conventions (by example) for test and tests helper:</simpara> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>project</simpara> |
| </entry> |
| <entry> |
| <simpara>-</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>project.tests</simpara> |
| </entry> |
| <entry> |
| <simpara>tests for project, is a fragment</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>project.tests.helper</simpara> |
| </entry> |
| <entry> |
| <simpara>helper classes used ONLY by tests</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>project.tests.performance</simpara> |
| </entry> |
| <entry> |
| <simpara>performance tests</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>project.tests.integration</simpara> |
| </entry> |
| <entry> |
| <simpara>integration tests</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>project.ui</simpara> |
| </entry> |
| <entry> |
| <simpara>-</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>project.ui.tests</simpara> |
| </entry> |
| <entry> |
| <simpara>tests for ui project, fragment of project.ui</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>project.ui.tests.helper</simpara> |
| </entry> |
| <entry> |
| <simpara>helper classes used ONLY by tests</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>project.ui.tests.performance</simpara> |
| </entry> |
| <entry> |
| <simpara>-</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>tests.helper</simpara> |
| </entry> |
| <entry> |
| <simpara>general test helper</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>ui.tests.helper</simpara> |
| </entry> |
| <entry> |
| <simpara>general ui test helper</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>project.xpect.tests</simpara> |
| </entry> |
| <entry> |
| <simpara>xpect tests for the project, despite dependnecies to UI the can be executed as plain JUnit tests</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>project.xpect.ui.tests</simpara> |
| </entry> |
| <entry> |
| <simpara>xpect tests for the project, need to be executed as eclipse plugin tests</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| <simpara>Due to Maven, tests are in subfolder tests (incl. helpers), implementation bundles in plugins, and release engineering related bundles in releng.</simpara> |
| </section> |
| </section> |
| </chapter> |
| <chapter xml:id="_eclipse-setup"> |
| <title>Eclipse Setup</title> |
| <section xml:id="_system-requirements"> |
| <title>System Requirements</title> |
| <simpara>In all cases, <link xl:href="https://adoptopenjdk.net/">Java 11</link> is required to be installed on your system. <link xl:href="https://nodejs.org/en/download/">Node.js</link> version 10+ is also required, and for some tests you need <link xl:href="https://yarnpkg.com">Yarn</link> to be globally installed.</simpara> |
| </section> |
| <section xml:id="_contribute"> |
| <title>Contribute</title> |
| <simpara>Eclipse developers who want to develop N4JS itself should use the <link xl:href="https://www.eclipse.org/downloads/">Oomph Eclipse installer</link>. The N4JS project is listed under "Eclipse Projects/N4JS" |
| This setup installs the correct Eclipse version, creates a new workspace and clones all projects into it (for details see below).</simpara> |
| <section xml:id="_eclipse-installer"> |
| <title>Eclipse Installer</title> |
| <simpara>The recommended way to install the Eclipse IDE and set up the workspace is to use the Eclipse Installer. |
| This installer is to be downloaded from <link xl:href="https://wiki.eclipse.org/Eclipse_Installer">https://wiki.eclipse.org/Eclipse_Installer</link></simpara> |
| <simpara>Run the installer and apply the following steps:</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>change to "Advance Mode" via the menu (upper-right corner) (no need to move the installer)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>select a product, e.g. "Eclipse IDE for Eclipse Committers" with product version "latest"</simpara> |
| </listitem> |
| <listitem> |
| <simpara>double-click the entry <emphasis role="strong">Eclipse Projects/N4JS</emphasis> so that it is shown in the catalog view below</simpara> |
| </listitem> |
| <listitem> |
| <simpara>on the next page, configure paths accordingly. You only have to configure the installation and workspace folder. You may want to use git with https instead of ssh.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>start installation</simpara> |
| </listitem> |
| </orderedlist> |
| <simpara>The installer will then guide you through the rest of the installation. All plug-ins are downloaded and configured automatically, so is the workspace including downloading the git repository and setting up the workspace.</simpara> |
| <simpara>The workspace is configured automatically. This includes fetching the necessary git repository. If you have selected git with SSH you may run into problems. In this case you can re-run the scripts and select HTTPS instead, this should work in any case.</simpara> |
| <simpara>Eventually the installer scripts are done, that means the git repository has been cloned and the workspace has been configured (including the project set setup). |
| Now the automatic build kicks in as you can see in the status bar. Screenshot 6</simpara> |
| <simpara>The build will show a lot of errors while still working. Eventually the whole project should have been compiled without any errors. Unfortunately, due to a <link xl:href="https://github.com/eclipse/n4js/issues/1373">known issue</link>, two problems exists. Please have a look at the linked issue on how to fix that (it is quite easy).</simpara> |
| <section xml:id="_changing-the-setup-script"> |
| <title>Changing the Setup Script</title> |
| <simpara>The setup scripts is stored at</simpara> |
| <simpara><literal>n4js/releng/org.eclipse.n4js.targetplatform/N4JS.setup</literal></simpara> |
| <simpara>Details about Oomph-Setup scripts can be found at</simpara> |
| <simpara><link xl:href="https://wiki.eclipse.org/Eclipse_Installer">https://wiki.eclipse.org/Eclipse_Installer</link></simpara> |
| </section> |
| </section> |
| <section xml:id="_manual-ide-configuration"> |
| <title>Manual IDE Configuration</title> |
| <warning> |
| <simpara>Manual IDE configuration is not recommended!</simpara> |
| </warning> |
| <simpara>For a manual install, clone the code and import all top-level projects from the docs, features, plugins, releng, testhelpers, and tests folders. Activate the targetplatform contained in the <literal>releng/org.eclipse.n4js.targetplatform/</literal> project.</simpara> |
| <simpara>The N4JS IDE is developed with Eclipse 2019-06 or better since the system is based on Eclipse anyway. |
| It is almost impossible to use another IDE to develop Eclipse plugins. The list of required plugins includes:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>Xtext/Xtend 2.18.0</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Xcore 1.9.0</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Xpect 0.2.0.201906240918 from <link xl:href="https://ci.eclipse.org/xpect/job/Xpect-Integration-Release/20/artifact/org.eclipse.xpect.releng/p2-repository/target/repository/">https://ci.eclipse.org/xpect/job/Xpect-Integration-Release/20/artifact/org.eclipse.xpect.releng/p2-repository/target/repository/</link></simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>It is important to use the latest version of Xtext and the corresponding service release of Xcore. You will find the latest version numbers and plugins used in the target platform definition at |
| <link xl:href="https://github.com/eclipse/n4js/blob/master/releng/org.eclipse.n4js.targetplatform/org.eclipse.n4js.targetplatform.target">https://github.com/eclipse/n4js/blob/master/releng/org.eclipse.n4js.targetplatform/org.eclipse.n4js.targetplatform.target</link></simpara> |
| <simpara>You may need to adjust some settings in Eclipse, most importantly</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><emphasis role="strong">Text file encoding</emphasis> to <literal>Other: UTF-8</literal> and</simpara> |
| </listitem> |
| <listitem> |
| <simpara><emphasis role="strong">New text file line delimiter</emphasis> to <literal>Unix</literal> .</simpara> |
| </listitem> |
| </itemizedlist> |
| </section> |
| </section> |
| </chapter> |
| <chapter xml:id="_release-engineering"> |
| <title>Release Engineering</title> |
| <section xml:id="_nightly-build-on-eclipse-infrastructure"> |
| <title>Nightly build on Eclipse infrastructure</title> |
| <simpara>The N4JS IDE, headless n4jsc.jar, and the N4JS update site is being built on the Eclipse Common Build |
| Infrastructure (CBI). For this purpose the N4JS project is using a dedicated Jenkins instance, referred |
| to as a "Jenkins Instance Per Project" (JIPP) in Eclipse CBI documentation. At this time, the N4JS |
| project’s JIPP is running on the "old" infrastructure, not yet using docker. This will be migrated |
| at a later point in time.</simpara> |
| <simpara>The N4JS JIPP is available at: <link xl:href="https://ci.eclipse.org/n4js/">https://ci.eclipse.org/n4js/</link></simpara> |
| <simpara>The nightly build performs the following main steps:</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>compile the N4JS implementation,</simpara> |
| </listitem> |
| <listitem> |
| <simpara>build the n4jsc.jar, the IDE products for MacOS, Windows, Linux, and the update site,</simpara> |
| </listitem> |
| <listitem> |
| <simpara>run tests,</simpara> |
| </listitem> |
| <listitem> |
| <simpara>sign the IDE product for macOS and package it in a .dmg file,</simpara> |
| </listitem> |
| <listitem> |
| <simpara>deploy to n4jsc.jar, IDE products and update sites to Eclipse download server (i.e. download.eclipse.org),</simpara> |
| </listitem> |
| <listitem> |
| <simpara>move all artifacts older than 7 days from download.eclipse.org to archive.eclipse.org.</simpara> |
| </listitem> |
| </orderedlist> |
| <simpara>Details about all the above steps can be found in the Jenkinsfile <literal>eclipse-nightly.jenkinsfile</literal>, located in |
| the root folder of the N4JS source repository on GitHub.</simpara> |
| <simpara>The most accurate documentation for our JIPP can be found at <link xl:href="https://wiki.eclipse.org/IT_Infrastructure_Doc">https://wiki.eclipse.org/IT_Infrastructure_Doc</link>. |
| Note that many other documents do not apply to our JIPP, at the moment, as they refer to the new |
| infrastructure, e.g. <link xl:href="https://wiki.eclipse.org/CBI">https://wiki.eclipse.org/CBI</link> and <link xl:href="https://wiki.eclipse.org/Jenkins">https://wiki.eclipse.org/Jenkins</link>.</simpara> |
| </section> |
| <section xml:id="_build-the-n4js-ide-from-command-line"> |
| <title>Build the N4JS IDE from command line</title> |
| <simpara>Ensure you have</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>Java 11</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Maven 3.2.x and</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Node.js 8</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>installed on your system.</simpara> |
| <simpara>Clone the repository</simpara> |
| <screen>git clone https://github.com/Eclipse/n4js.git</screen> |
| <simpara>Change to the n4js folder:</simpara> |
| <screen>cd n4js</screen> |
| <simpara>Run the Maven build:</simpara> |
| <screen>mvn clean verify</screen> |
| <simpara>You may have to increase the memory for maven via <literal>export MAVEN_OPTS="-Xmx2048m"</literal> (Unix) or <literal>set MAVEN_OPTS="-Xmx2048m"</literal> (Windows).</simpara> |
| <simpara>Available optional maven profiles are:</simpara> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>buildProduct</simpara> |
| </entry> |
| <entry> |
| <simpara>create IDE products (Windows, macOS, Linux) and a jar for headless compilation</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>execute-plugin-tests</simpara> |
| </entry> |
| <entry> |
| <simpara>run OSGi tests (without UI)</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>execute-plugin-ui-tests</simpara> |
| </entry> |
| <entry> |
| <simpara>run UI-based OSGi tests</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>execute-ecmas-tests</simpara> |
| </entry> |
| <entry> |
| <simpara>run ECMA test suite</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>execute-smoke-tests</simpara> |
| </entry> |
| <entry> |
| <simpara>run generated tests using corrupted source code as input</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>execute-accesscontrol-tests</simpara> |
| </entry> |
| <entry> |
| <simpara>run generated tests for checking accessibility of class/interface members</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>execute-hlc-integration-tests</simpara> |
| </entry> |
| <entry> |
| <simpara>run integration tests using the headless jar (requires docker!)</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| <simpara>Available system properties:</simpara> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>noTests</simpara> |
| </entry> |
| <entry> |
| <simpara>suppress execution of all tests</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>startAndKeepVerdaccio</simpara> |
| </entry> |
| <entry> |
| <simpara>enforce starting and suppress stopping of the test verdaccio (see <xref linkend="sec:test-verdaccio"/>)</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| <section xml:id="_publish-maven-tooling-literal-org-eclipse-n4js-releng-util-literal"> |
| <title>Publish maven-tooling <literal>org.eclipse.n4js.releng.util</literal></title> |
| <note> |
| <simpara>For extending the N4JS-language in a different project, the <literal>org.eclipse.n4js.releng.util</literal> module needs to be published as a maven-plugin. You can deploy this SNAPSHOT-artifact to a local folder by providing the <literal>local-snapshot-deploy-folder</literal>-property pointing to an absolute path in the local file system:</simpara> |
| </note> |
| <screen>mvn clean deploy -Dlocal-snapshot-deploy-folder=/var/lib/my/folder/local-mvn-deploy-repository</screen> |
| <simpara>The existence of <literal>local-snapshot-deploy-folder</literal> will trigger a profile enabling the deploy-goal for the project <literal>org.eclipse.n4js.releng.util</literal></simpara> |
| </section> |
| <section xml:id="sec:test-verdaccio"> |
| <title>Test Verdaccio containing n4js-libs</title> |
| <simpara>If profile <literal>execute-hlc-integration-tests</literal> is active, a local verdaccio instance is started and populated with |
| freshly-compiled n4js-libs (the libraries located under top-level folder <literal>/n4js-libs</literal>) and is stopped before the |
| end of the build. The verdaccio instance is started as a docker container called <literal>n4js-test-verdaccio</literal>.</simpara> |
| <simpara>When giving <literal>-DstartAndKeepVerdaccio</literal> on the command line, such a test verdaccio will always be started/populated but |
| never stopped, regardless of whether profile <literal>execute-hlc-integration-tests</literal> is active or not. This is useful to enforce |
| starting of the test verdaccio (even without running integration tests) and then reusing it in subsequent builds.</simpara> |
| </section> |
| <section xml:id="_generation-of-eclipse-help-for-spec-and-design-document"> |
| <title>Generation of Eclipse help for spec and design document</title> |
| <simpara>The HTML pages for N4JSSpec and N4JSDesign documents are generated from the Asciidoc sources in the project <literal>org.eclipse.n4js.spec</literal> <literal>org.eclipse.n4js.design</literal> by Asciispec. </simpara> |
| <figure xml:id="img:eclipse-help-doc-process"> |
| <title>The process of creating Eclipse help for N4JSSpec</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/03_releng/images/eclipse-help-process.svg"/> |
| </imageobject> |
| <textobject><phrase>Creating Eclipse help for N4JSSpec</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara>Figure <xref linkend="img:eclipse-help-doc-process"/> shows the generation process for N4JSSpec document. The process for N4JSDesign (and other adoc documents) is the same. The following explains the diagram.</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><literal>Asciispec</literal> is used to compile the source N4JSSpec Asciidoc into a single large <literal>N4JSSpec.html</literal> file which contains all the chapters. The use of the custom parameter <literal>-a eclipse-help-mode</literal> indicates that a special header and footer styles as well as CSS style should be used (i.e. no table of content menu, no download links etc.). Here, we are using the possibility provided by Asciidoctor to configure header/footer as well as CSS style via parameter <literal>:docinfodir:</literal> and <literal>:stylesheet:</literal>.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Our custom tool <literal>Chunker</literal> splits <literal>N4JSSpec.html</literal> (and other documents) into multiple chunked HTML files, each of which corresponds to either the <literal>index</literal> file or a chapter. It automatically re-writes internal links.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Another custom tool <literal>EclipseHelpTOCGenerator</literal> takes to Docbook file <literal>N4JSSpec.xml</literal> and generates an XML file describing the table of content (TOC) in the Eclipse format. This TOC file references the chunked HTML files above.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Another custom tool <literal>IndexTocGenerator</literal> takes to Docbook file <literal>N4JSSpec.xml</literal> similar to <literal>EclipseHelpTOCGenerator</literal>, but it generates an HTML fragment which can be embedded into the <literal>index.html</literal> page generated by the <literal>Chunker</literal> (Thus it has to run before the Chunker in that case).</simpara> |
| </listitem> |
| </itemizedlist> |
| </section> |
| </section> |
| <section xml:id="_updating-frameworks-and-dependencies"> |
| <title>Updating frameworks and dependencies</title> |
| <section xml:id="_update-of-eclipse-emf-xtext-etc"> |
| <title>Update of Eclipse, EMF, Xtext, etc.</title> |
| <simpara>For updating the N4JS IDE to a new version of Eclipse, EMF, Xtext, etc. follow these steps:</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>Create a new branch.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Bump versions of all dependencies mentioned in file <literal>N4JS.setup</literal>:</simpara> |
| <orderedlist numeration="loweralpha"> |
| <listitem> |
| <simpara>Update all labels that refer to the version of the Ooomph setup (search for "label!" to find them).</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Choose a new Eclipse version and define this in <literal>N4JS.setup</literal>.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>For those other dependencies <emphasis>that come with Eclipse</emphasis> (e.g. EMF, Xtext) find out which version matches the chosen Eclipse version |
| and define that version in <literal>N4JS.setup</literal>.<?asciidoc-br?> |
| Tip: use the contents list of the SimRel you are targeting, e.g. <link xl:href="https://projects.eclipse.org/releases/2019-03">https://projects.eclipse.org/releases/2019-03</link></simpara> |
| </listitem> |
| <listitem> |
| <simpara>For those other dependencies <emphasis>that are available via the Eclipse Orbit</emphasis>, find out which version is the latest version available in |
| the Orbit and define that version in <literal>N4JS.setup</literal>.<?asciidoc-br?> |
| Tip: contents of the Eclipse Orbit can be found at <link xl:href="https://download.eclipse.org/tools/orbit/downloads/">https://download.eclipse.org/tools/orbit/downloads/</link><?asciidoc-br?> |
| (choose the correct link for the chosen Eclipse version!)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>For all remaining dependencies (i.e. unrelated to Eclipse and not in Orbit), choose a version to use and define it in <literal>N4JS.setup</literal>.</simpara> |
| </listitem> |
| </orderedlist> |
| </listitem> |
| <listitem> |
| <simpara>Check <literal>Require-Bundle</literal> sections of MANIFEST.MF files by searching for related bundle names or for <literal>;bundle-version="</literal>:</simpara> |
| <orderedlist numeration="loweralpha"> |
| <listitem> |
| <simpara>There should be at most one version constraint for a specific bundle<?asciidoc-br?> |
| NOTE: the version constraints in the MANIFEST.MF files are just lower bounds and - at this time - we do not bump them to the latest version, in most cases.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>There should be no version constraints to our bundles (i.e. <literal>org.eclipse.n4js…​</literal>)</simpara> |
| </listitem> |
| </orderedlist> |
| </listitem> |
| <listitem> |
| <simpara>Review parent pom.xml files, i.e. <literal>releng/org.eclipse.n4js.parent/pom.xml</literal>:</simpara> |
| <orderedlist numeration="loweralpha"> |
| <listitem> |
| <simpara>Update property <literal>xtext-version</literal>.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Check all other <literal>*-version</literal> properties and update them where needed.</simpara> |
| </listitem> |
| </orderedlist> |
| </listitem> |
| <listitem> |
| <simpara>Update target platform file <literal>org.eclipse.n4js.targetplatform.target</literal> using Ooomph’s auto-generation:</simpara> |
| <orderedlist numeration="loweralpha"> |
| <listitem> |
| <simpara>Start the Eclipse Installer.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Update the Eclipse Installer (using the button with the turning arrows).</simpara> |
| </listitem> |
| <listitem> |
| <simpara>On the second page, add the <literal>N4JS.setup</literal> file from your branch to the Eclipse Installer, using a GitHub raw(!) URL:<?asciidoc-br?> |
| <literal><link xl:href="https://raw.githubusercontent.com/eclipse/n4js/BRANCH_NAME/releng/org.eclipse.n4js.targetplatform/N4JS.setup">https://raw.githubusercontent.com/eclipse/n4js/BRANCH_NAME/releng/org.eclipse.n4js.targetplatform/N4JS.setup</link></literal></simpara> |
| </listitem> |
| <listitem> |
| <simpara>Ooomph a new development environment with this setup.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>In the new Eclipse workspace created by Ooomph, the target platform file should have uncommitted changes:</simpara> |
| <orderedlist numeration="lowerroman"> |
| <listitem> |
| <simpara>carefully review these changes, to be sure they make sense, and then</simpara> |
| </listitem> |
| <listitem> |
| <simpara>commit & push those changes to your branch.</simpara> |
| </listitem> |
| </orderedlist> |
| </listitem> |
| </orderedlist> |
| </listitem> |
| <listitem> |
| <simpara>Thoroughly test the new versions, including some manual(!) tests:</simpara> |
| <orderedlist numeration="loweralpha"> |
| <listitem> |
| <simpara>Run Jenkins builds.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Ooomph another N4JS development environment with Eclipse Installer. |
| This time, after Ooomphing is completed, the target platform file should no longer have any uncommitted changes.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Ensure the following types of tests can be executed locally in the newly installed Eclipse:</simpara> |
| <orderedlist numeration="lowerroman"> |
| <listitem> |
| <simpara>plain JUnit tests (e.g. <literal>org.eclipse.n4js.lang.tests</literal>).</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Plugin tests.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Plugin UI tests.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>SWTBot tests.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Xpect tests (individual files and entire bundles; e.g. <literal>org.eclipse.n4js.spec.tests</literal>).</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Xpect UI tests.</simpara> |
| </listitem> |
| </orderedlist> |
| </listitem> |
| <listitem> |
| <simpara>Ensure an N4JS IDE product can be launched from within the newly installed Eclipse using the launch configuration |
| provided in the n4js repository.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>After launching the N4JS IDE product, refresh the workspace and review/commit any changes in file <literal>N4JS__IDE.launch</literal>.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Download a product created in a Jenkins CI build and test it manually.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>After merging to master: download a product created in a nightly build and test it manually. |
| Ensure signing and JRE bundling are still working properly.</simpara> |
| </listitem> |
| </orderedlist> |
| </listitem> |
| </orderedlist> |
| <simpara>All the above steps need to be performed in the <literal>n4js-n4</literal> repository, accordingly (e.g. file <literal>N4JS-N4.setup</literal>).</simpara> |
| </section> |
| <section xml:id="_update-of-the-embedded-jre"> |
| <title>Update of the embedded JRE</title> |
| <simpara>For updating the embedded JRE inside the N4JS IDE follow these steps:</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>Given a new JRE download location for Linux, MacOS and Windows with a common new version</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Update the location related properties in the pom.xml files of</simpara> |
| <orderedlist numeration="loweralpha"> |
| <listitem> |
| <simpara>n4js/builds/pom.xml</simpara> |
| </listitem> |
| <listitem> |
| <simpara>n4js/builds/org.eclipse.n4js.jre.linux.gtk.x86_64/pom.xml</simpara> |
| </listitem> |
| <listitem> |
| <simpara>n4js/builds/org.eclipse.n4js.jre.macosx.cocoa.x86_64/pom.xml</simpara> |
| </listitem> |
| <listitem> |
| <simpara>n4js/builds/org.eclipse.n4js.jre.win32.win32.x86_64/pom.xml</simpara> |
| </listitem> |
| </orderedlist> |
| </listitem> |
| <listitem> |
| <simpara>Update the versions at all following locations:</simpara> |
| <orderedlist numeration="loweralpha"> |
| <listitem> |
| <simpara>n4js/builds/org.eclipse.n4js.jre.linux.gtk.x86_64/META-INF/MANIFEST.MF</simpara> |
| </listitem> |
| <listitem> |
| <simpara>n4js/builds/org.eclipse.n4js.jre.linux.gtk.x86_64/META-INF/p2.inf</simpara> |
| </listitem> |
| <listitem> |
| <simpara>n4js/builds/org.eclipse.n4js.jre.macosx.cocoa.x86_64/META-INF/MANIFEST.MF</simpara> |
| </listitem> |
| <listitem> |
| <simpara>n4js/builds/org.eclipse.n4js.jre.macosx.cocoa.x86_64/META-INF/p2.inf</simpara> |
| </listitem> |
| <listitem> |
| <simpara>n4js/builds/org.eclipse.n4js.jre.win32.win32.x86_64/META-INF/MANIFEST.MF</simpara> |
| </listitem> |
| <listitem> |
| <simpara>n4js/builds/org.eclipse.n4js.jre.win32.win32.x86_64/META-INF/p2.inf</simpara> |
| </listitem> |
| </orderedlist> |
| </listitem> |
| <listitem> |
| <simpara>Update the openjdk docker image used as base image in the "FROM" line at the top of all docker files:</simpara> |
| <orderedlist numeration="loweralpha"> |
| <listitem> |
| <simpara>n4js-n4/jenkins/docker-build/Dockerfile</simpara> |
| </listitem> |
| </orderedlist> |
| </listitem> |
| </orderedlist> |
| </section> |
| </section> |
| </chapter> |
| <chapter xml:id="_tips-and-tricks"> |
| <title>Tips and Tricks</title> |
| <simpara>In this chapter we collect some coding hints and guidelines on how to properly use the APIs of Eclipse, EMF, Xtext and |
| other dependencies we are using, as well as our own utilities and helpers.</simpara> |
| <simpara>This chapter is only about coding; add information on things like Eclipse setup or Maven/Jenkins to one of the preceding |
| chapters. Similarly, this chapter is intended to provide just a quick overview, check-list and reminder; add detailed |
| information and diagrams to one of the succeeding chapters.</simpara> |
| <section xml:id="_naming"> |
| <title>Naming</title> |
| <itemizedlist> |
| <listitem> |
| <simpara>The internal handling of N4JS project names is non-trivial (due to the support for npm scopes), see |
| API documentation of <literal>ProjectDescriptionUtils#isProjectNameWithScope(String)</literal> for a detailed overview. |
| In short:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><literal>IN4JSProject#getProjectName()</literal> and <literal>IProject#getName()</literal> return different values!</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Avoid using the Eclipse project name, i.e. the return value of <literal>IProject#getName()</literal>, as far as possible |
| (only use it in UI code when actually dealing with what is shown in the Eclipse UI).</simpara> |
| </listitem> |
| <listitem> |
| <simpara>The last segment of an URI or path pointing to an N4JS project is <emphasis role="strong">not</emphasis> always the project name; use |
| utilities in <literal>ProjectDescriptionUtils</literal> instead, e.g. <literal>#deriveN4JSProjectNameFromURI()</literal>! |
| (However, given an URI or path pointing to a file inside an N4JS project, you can use its last segment |
| to obtain the file name.)</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| </itemizedlist> |
| </section> |
| <section xml:id="_logging"> |
| <title>Logging</title> |
| <simpara>In many situations developer needs to use some kind of logging. When in need, follow these rules:</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>Use <literal>org.apache.log4j.Logger;</literal> for logging. Other logging utilities (like java built in logger) are not configured.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>do not use <literal>System.out</literal> nor <literal>Sysetem.err</literal> for logging. It is ok to use it for debugging purposes, but those calls |
| should never be merged to master. <emphasis>(with exception of headless compiler, which uses them explicitly)</emphasis></simpara> |
| </listitem> |
| <listitem> |
| <simpara>There is central logger configuration in <literal>org.eclipse.n4js.utils.logging</literal> (and <literal>org.eclipse.n4js.utils.logging</literal>) that should |
| be used</simpara> |
| <orderedlist numeration="loweralpha"> |
| <listitem> |
| <simpara><literal>log4j.xml</literal> used for production</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>log4j_tests.xml</literal> used when running tests</simpara> |
| </listitem> |
| </orderedlist> |
| </listitem> |
| <listitem> |
| <simpara>in Eclipse run configurations logger has to be set properly, e.g. |
| <literal>log4j.configuration=file:${workspace_loc:org.eclipse.n4js.utils.logging/log4j_tests.xml}</literal></simpara> |
| </listitem> |
| <listitem> |
| <simpara>in maven configurations logger has to be set separately, e.g. |
| <literal>-Dlog4j.configuration="file:${basedir}/../../plugins/org.eclipse.n4js.utils.logging/log4j_tests.xml</literal></simpara> |
| </listitem> |
| </orderedlist> |
| </section> |
| <section xml:id="_cancellation-handling"> |
| <title>Cancellation Handling</title> |
| <simpara>At various occasions, Xtext provides an instance of class <literal>CancelIndicator</literal> to allow our code to handle cancellation of |
| long-running task.</simpara> |
| <simpara>Some things to keep in mind:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>whenever a <literal>CancelIndicator</literal> is available any code that might not return immediately should implement proper |
| cancellation handling (as explained in the next items).</simpara> |
| </listitem> |
| <listitem> |
| <simpara>most importantly: reacting to a cancellation by returning early from a method is an anti-pattern that leads to |
| problems (client code might continue work on a canceled and thus invalid state); instead: throw an |
| <literal>OperationCanceledException</literal>!</simpara> |
| </listitem> |
| <listitem> |
| <simpara>don’t use <literal>CancelIndicator#isCanceled()</literal> for cancellation handling, except in certain special cases. A valid exception |
| case might be during logging to show a message like "operation was canceled".</simpara> |
| </listitem> |
| <listitem> |
| <simpara>instead, inject the Xtext service called <literal>OperationCanceledManager</literal> and invoke its method <literal>#checkCanceled()</literal>, passing-in |
| the cancel indicator (this method is null-safe; it will throw an <literal>OperationCanceledException</literal> in case a cancellation has |
| occurred). Don’t directly create and throw an <literal>OperationCanceledException</literal> yourself.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>use the other methods provided by <literal>OperationCanceledManager</literal> when appropriate (see code of that class for details).</simpara> |
| </listitem> |
| <listitem> |
| <simpara>in try/catch blocks, when catching exceptions of a super type of <literal>OperationCanceledException</literal>, be sure to <emphasis role="strong">not suppress</emphasis> |
| cancellation exceptions. For example:</simpara> |
| <programlisting language="java" linenumbering="unnumbered">// Java code |
| @Inject private OperationCanceledManager operationCanceledManager; |
| /** Returns true on success, false otherwise. */ |
| public boolean doSomething(CancelIndicator ci) { |
| try { |
| // do something that might be canceled |
| return true; |
| } catch(Exception e) { |
| operationCanceledManager.propagateIfCancelException(e); // <- IMPORTANT! |
| return false; |
| } |
| }</programlisting> |
| <simpara>Try/finally blocks, on the other hand, do not need any special handling.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>a cancel indicator can also be stored in the rule environment (see <literal>RuleEnvironmentExtensions#addCancelIndicator()</literal>). This |
| means:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>if you create a rule environment completely from scratch and you have a cancel indicator at hand, add it to the rule |
| environment via <literal>RuleEnvironmentExtensions#addCancelIndicator()</literal> (not required when using <literal>RuleEnvironmentExtensions#wrap()</literal> for |
| deriving a rule environment from an existing one).</simpara> |
| </listitem> |
| <listitem> |
| <simpara>if you have a rule environment available, be sure to use its cancel indicator in long-running operations, i.e. with |
| code like:</simpara> |
| <programlisting language="java" linenumbering="unnumbered">// Xtend code |
| import static extension org.eclipse.n4js.typesystem.utils.RuleEnvironmentExtensions.* |
| class C { |
| @Inject private OperationCanceledManager operationCanceledManager; |
| def void doSomething() { |
| for(a : aLotOfStuff) { |
| operationCanceledManager.checkCanceled(G.cancelIndicator); |
| // main work ... |
| } |
| }</programlisting> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| </itemizedlist> |
| </section> |
| <section xml:id="_caching"> |
| <title>Caching</title> |
| <itemizedlist> |
| <listitem> |
| <simpara>Caching of external libraries (implemented in ExternalProjectMappings)</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>update <emphasis>only</emphasis> using <literal>EclipseExternalLibraryWorkspace#updateState()</literal></simpara> |
| </listitem> |
| <listitem> |
| <simpara>always mind that the diff of current state and cached state is a necessary information for cleaning dependencies of removed npms</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>see <literal>EclipseExternalIndexSynchronizer#synchronizeNpms()</literal> for implementation</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara>updating also happens when external root locations change (see ExternalIndexUpdater)</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara>Caching of user workspace projects (implemented in MuliCleartriggerCache)</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>caches only some project information and should be refactored along with Core, Model and EclipseBasedN4JSWorkspace</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| </itemizedlist> |
| </section> |
| <section xml:id="_dependency-injection"> |
| <title>Dependency Injection</title> |
| <simpara>There are some things to keep in mind when using dependency injection in the context of Xtext. This is a longer topic and it is discussed in the appendix |
| <xref linkend="sec:XtextInjection"/>.</simpara> |
| </section> |
| <section xml:id="_miscellaneous"> |
| <title>Miscellaneous</title> |
| <itemizedlist> |
| <listitem> |
| <simpara>Resource load states: when an N4JS/N4JSD file is loaded, a certain sequence of processing is triggered (parsing, |
| linking, validation, etc.) and thus an <literal>N4JSResource</literal> transitions through a sequence of "load states". For details, |
| see <xref linkend="sec:N4JS_Resource_Load_States"/>.</simpara> |
| </listitem> |
| </itemizedlist> |
| </section> |
| </chapter> |
| <chapter xml:id="_parser"> |
| <title>Parser</title> |
| <simpara>Some of the concepts described here were presented at |
| <link xl:href="https://www.youtube.com/watch?v=Xm-7aE1UMGY&feature=youtu.be">EclipseCon 2013</link> and |
| <link xl:href="https://vimeo.com/channels/xtextcon/98446435">XtextCon 2014</link>. Note that the material presented at the linked videos may be outdated.</simpara> |
| <section xml:id="sec:Parser_Overview"> |
| <title>Overview</title> |
| <simpara>The parser is created from an Xtext grammar. Actually, there are several grammars used as shown in <link linkend="fig:cd_grammars">Figure CD Grammars</link>. These grammars and the parsers generated from them are described more closely in the following sections.</simpara> |
| <figure xml:id="fig:cd_grammars" role="center"> |
| <title>N4 Grammars</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/05_parser/images/cd_grammars.svg"/> |
| </imageobject> |
| <textobject><phrase>cd grammars</phrase></textobject> |
| </mediaobject> |
| </figure> |
| </section> |
| <section xml:id="sec:N4JS_Parser"> |
| <title>N4JS Parser</title> |
| <simpara>One of the most tricky parts of JavaScript is the parsing because there is a conceptual mismatch between the <link linkend="AC">ANTLR</link> runtime and the specified grammar. Another challenge is the disambiguation of regular expressions and binary operations. Both features require significant customizing of the generated parser (see figure below).</simpara> |
| <figure xml:id="fig:cd_ASIParser" role="center"> |
| <title>Overview custom parser implementation (runtime only)</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/05_parser/images/cd_ASIParser.svg"/> |
| </imageobject> |
| <textobject><phrase>cd ASIParser</phrase></textobject> |
| </mediaobject> |
| </figure> |
| </section> |
| <section xml:id="sec:Parser_Generation_Post_Processing" role="language-bash"> |
| <title>Parser Generation Post-Processing</title> |
| <simpara>The ANTLR grammar that is generated by Xtext is post-processed to inject custom code into the grammar file before it is passed to the ANTLR tool. This is required in particular due to <link linkend="AC">ASI</link> (Automated Semicolon Insertion), but for some other reasons as well.</simpara> |
| <simpara>Actually, there are several injections:</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>Due to Xtext restrictions, the generated ANTLR grammar file (<emphasis role="strong">*.g</emphasis>) is modified. This means that some some additional actions are added and some rules are rewritten.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Due to ANTLR restrictions, the generated ANTLR Java parser (<emphasis role="strong">*.java</emphasis>) os modified. This means that some generated rules are slightly modified to match certain requirements.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Due to Java restrictions, the generated Java parser needs to be preprocessed in order to reduce the size of certain methods since they must not exceed 64k characters. This is implemented by means of an MWE fragment, activated after the other post processing steps are done.</simpara> |
| </listitem> |
| </orderedlist> |
| <simpara>The first two steps are handled by <literal>AntlrGeneratorWithCustomKeywordLogic</literal>, which is configured with additional helpers in <literal>GenerateN4JS.mwe2</literal>. shows the customized classes which modify the code generation. These classes are all part of the <literal>releng.utils</literal> bundle.</simpara> |
| <figure role="center"> |
| <title>Class Diagram Parser Generation</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/05_parser/images/cd_parsergeneration.svg"/> |
| </imageobject> |
| <textobject><phrase>cd parsergeneration</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <section xml:id="sec:Automatic_Semicolon_Insertion"> |
| <title>Automatic Semicolon Insertion</title> |
| <simpara>The EcmaScript specification mandates that valid implementations automatically insert a semicolon as a statement delimiter if it is missing and the input file would become invalid due to the missing semicolon. This is known as <link linkend="AC">ASI</link>. It implies that not only valid implementations have to perform this, but a valid parser has to mimic this behavior in order to parse executable code. The <link linkend="AC">ASI</link> is implemented by two different means.</simpara> |
| <simpara>The parser’s error recovery strategy is customized so it attempts to insert a semicolon if it was expected. Both strategies have to work hand in hand in order to consume all sorts of legal JavaScript code.</simpara> |
| <section xml:id="sec:Injected_code_in_the_Antlr_grammar_file"> |
| <title>Injected code in the Antlr grammar file</title> |
| <simpara>Under certain circumstances, the parser has to actively promote a token to become a semicolon even though it may be a syntactically a closing brace or line break. This has to happen before that token is consumed thus the rules for return statements, continue statements and break statements are enhanced to actively promote these tokens to semicolons.</simpara> |
| <simpara>The same rule is applied to promote line breaks between an expression and a possible postfix operator <literal>++</literal> or <literal>–</literal>. At this location the line break is always treated as a semicolon even though the operator may be validly consumed and produce a postfix expression.</simpara> |
| <simpara>In both cases, the method <literal>promoteEOL()</literal> is used to move a token that may serve as an automatically injected semicolon from the so called hidden token channel to the semantic channel. The hidden tokens are usually not handled by the parser explicitly thus they are semantically invisible (therefore the term hidden token). Nevertheless, they can be put on the semantic channel explicitly to make them recognizable. That’s implemented in the EOL promotion. The offending tokens include the hidden line terminators and multi-line comments that include line breaks. Furthermore, closing braces (right curly brackets) are included in the set of offending tokens as well as explicit semicolons.</simpara> |
| </section> |
| <section xml:id="sec:Customized_error_recovery"> |
| <title>Customized error recovery</title> |
| <simpara>Since the EOL promotion does not work well with Antlr prediction mode, another customization complements that feature. As soon as an invalid token sequence is attempted to be parsed and missing semicolon would make that sequence valid, an offending token is sought and moved to the semantic channel. This is implemented in the custom recovery strategy.</simpara> |
| </section> |
| </section> |
| <section xml:id="sec:_No_line_terminator_allowed_here__handling"> |
| <title>Async and <literal>No line terminator allowed here</literal> Handling</title> |
| <simpara>There is no way of directly defining <literal>No line terminator allowed here</literal>. This is required not only for <link linkend="AC">ASI</link>, but also for <literal>async</literal>. This requires not only a special rule (using some rules from <link linkend="sec:Automatic_Semicolon_Insertion">ASI</link>), but also a special error recovery since the token ’async’ may be rejected (by the manually enriched rule) which is of course unexpected behavior from the generated source code.</simpara> |
| </section> |
| <section xml:id="sec:Regular_Expression"> |
| <title>Regular Expression</title> |
| <simpara>The ANTLR parsing process can basically be divided into three steps. First of all, the file contents has to be read from disk. This includes the proper encoding of bytes to characters. The second step is the lexing or tokenizing of the character stream. A token is a basically a typed region in the stream, that is a triplet of token-id, offset and length. The last step is the parsing of these tokens. The result is a semantic model that is associated with a node tree. All necessary information to validate the model can be deduced from these two interlinked representations.</simpara> |
| <figure role="center"> |
| <title>Simplified visualization of the parsing</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/05_parser/images/ad_parsing_simplified.svg"/> |
| </imageobject> |
| <textobject><phrase>ad parsing simplified</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara>Since the default semantics and control flow of Antlr generated parsers do not really fit the requirements of a fully working JavaScript parser, some customizations are necessary. <emphasis role="strong">Regular expression literals in JavaScript cannot be syntactically disambiguated from div operations without contextual information.</emphasis> Nevertheless, the spec clearly describes, where a regular expression may appear and where it is prohibited. Unfortunately, it is not possible to implement these rules in the lexer alone, since it does not have enough contextual information. Therefore, the parser has been enhanced to establish a communication channel with the lexer. It announces when it expects a regular expression rather than a binary operation.</simpara> |
| <simpara>This required a reworking of the Antlr internals. Instead of a completely pre-populated <literal>TokenStream</literal>, the parser works on a lazy implementation that only reads as many characters as possible without a disambiguation between regular expression literals and divide operators.</simpara> |
| <simpara>Only after the parser has read this buffered tokens and potentially announced that it expects a regular expression, another batch of characters is processed by the lexer until the next ambiguous situation occurs. This is fundamentally different from the default behavior of Antlr.</simpara> |
| <figure role="center"> |
| <title>Abstract control and object flow during parsing</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/05_parser/images/sd_parsing_sequence.svg"/> |
| </imageobject> |
| <textobject><phrase>sd parsing sequence</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara>shows the involved classes which allow for this lexer-parser communication.</simpara> |
| <figure role="center"> |
| <title>Class Diagram Parser-Lexer Communication</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/05_parser/images/cd_parserlexercommunication.svg"/> |
| </imageobject> |
| <textobject><phrase>cd parserlexercommunication</phrase></textobject> |
| </mediaobject> |
| </figure> |
| </section> |
| <section xml:id="sec:Unicode"> |
| <title>Unicode</title> |
| <simpara>Unicode support in JavaScript includes the possibility to use unicode escape sequences in identifiers, string literals and regular expression literals. Another issue in this field is the specification of valid identifiers in JavaScript. They are described by means of unicode character classes. These have to be enumerated in the terminal rules in order to fully accept or reject valid or invalid JS identifiers.</simpara> |
| <simpara>For that purpose, a small code generator is used to define the terminal fragments for certain unicode categories. The <literal>UnicodeGrammarGenerator</literal> basically iterates all characters from <literal>Character.MIN_VALUE</literal> to <literal>Character.MAX_VALUE</literal> and adds them as alternatives to the respective terminal fragments, e.g. <literal>UNICODE_DIGIT_FRAGMENT</literal>.</simpara> |
| <simpara>The real terminal rules are defined as a composition of these generated fragments. Besides that, each character in an identifier, in a string literal or in a regular expression literal may be represented by its unicode escape value, e.g. ` u0060`. These escape sequences are handled and validated by the <literal>IValueConverter</literal> for the corresponding terminal rules.</simpara> |
| <simpara>The second piece of the puzzle are the unicode escaped sequences that may be used in keywords. This issue is covered by the <literal>UnicodeKeywordHelper</literal> which replaces the default terminal representation in the generated Antlr grammar by more elaborated alternatives. The keyword <literal>if</literal> is not only lexed as <literal>’if’</literal> but as seen in snippet |
| <link linkend="lst:terminal_if">Terminal if listing</link>.</simpara> |
| <formalpara xml:id="lst:terminal_if"> |
| <title>Terminal if</title> |
| <para> |
| <screen>If : |
| ( 'i' | '\\' 'u' '0`` 0`` 6`` 9' ) |
| ( 'f' | '\\' 'u' '0`` 0`` 6`` 6' );</screen> |
| </para> |
| </formalpara> |
| </section> |
| <section xml:id="sec:Literals"> |
| <title>Literals</title> |
| <simpara>Template literals are also to be handled specially, see <literal>TemplateLiteralDisambiguationInjector</literal> for details.</simpara> |
| </section> |
| </section> |
| <section xml:id="sec:Modifiers" role="language-n4js"> |
| <title>Modifiers</title> |
| <simpara>On the AST side, all modifiers are included in a single enumeration <literal>N4Modifier</literal>. In the types model however, the individual modifiers are mapped to two different enumerations of <emphasis>access</emphasis> modifiers (namely <literal>TypeAccessModifier</literal> and <literal>MemberAccessModifier</literal>) and a number of boolean properties (in case of non-access modifiers such as <literal>abstract</literal> or <literal>static</literal>). This mapping is done by the types builder, mostly by calling methods in class <literal>ModifierUtils</literal>.</simpara> |
| <simpara>The grammar allows the use of certain modifiers in many places that are actually invalid. Rules where a certain modifier may appear in the AST are implemented in method isValid(EClass,N4Modifier) in class <literal>ModifierUtils</literal> and checked via several validations in <literal>N4JSSyntaxValidator</literal>. Those validations also check for a particular order of modifiers that is not enforced by the grammar.</simpara> |
| <simpara>See API documentation of enumeration <literal>N4Modifier</literal> in file <literal>N4JS.xcore</literal> and the utility class <literal>ModifierUtils</literal> for more details.</simpara> |
| </section> |
| <section xml:id="sec:Conflict_Resolutions" role="language-n4js"> |
| <title>Conflict Resolutions</title> |
| <section xml:id="sec:Reserved_Keywords_vs__Identifier_Names"> |
| <title>Reserved Keywords vs. Identifier Names</title> |
| <simpara>Keywords and identifiers have to be distinguished by the lexer. Therefore, there is no means to decide upfront whether a certain keyword is actually used as a keyword or whether it is used as an identifier in a given context. This limitation is idiomatically overcome by a data type rule for valid identifiers. This data type rule enumerates all keywords which may be used as identifiers and the pure IDENTIFIER terminal rule as seen in <link linkend="lst:keywords_as_identifier">Keywords as Identifier listing</link>.</simpara> |
| <formalpara xml:id="lst:keywords_as_identifier"> |
| <title>Keywords as Identifier</title> |
| <para> |
| <programlisting language="ebnf" linenumbering="unnumbered">N4JSIdentifier: IDENTIFIER |
| | 'get' |
| | 'set' |
| ... |
| ;</programlisting> |
| </para> |
| </formalpara> |
| </section> |
| <section xml:id="sec:Operators_and_Generics"> |
| <title>Operators and Generics</title> |
| <simpara>The ambiguity between shift operators and nested generics arises also from the fact, that Antlr lexer upfront without any contextual information. When implemented naively, the grammar will be broken, since a token sequence <literal>a>>b</literal> can either be part of <literal>List<List<a>> b</literal> or it can be part of a binary operation <literal>int c = a >> b</literal>. Therefore the shift operator may not be defined with a single token but has to be composed from individual characters (see <link linkend="lst:shift_operator">Shift Operator listing</link>).</simpara> |
| <formalpara xml:id="lst:shift_operator"> |
| <title>Shift Operator listing</title> |
| <para> |
| <programlisting language="ebnf" linenumbering="unnumbered">ShiftOperator: |
| '>' '>' '>'? |
| | '<' '<' |
| ;</programlisting> |
| </para> |
| </formalpara> |
| </section> |
| </section> |
| <section xml:id="sec:Content_Assist_Parser" role="language-n4js"> |
| <title>Content-Assist Parser</title> |
| <warning> |
| <simpara>This section may be outdated!</simpara> |
| </warning> |
| <simpara>The <link linkend="AC">CA</link> parser also needs adjustments for supporting automatic semicolon insertion and regular expressions. Instead of modifying the <link linkend="AC">CA</link> parser generator similar to the normal parser, the former reuses parts of the latter as far as possible. That is, the token sequence that is produced during production parsing is used as is for the content assist parser. Semicolons have already been inserted where appropriate and regular expression are successfully distinguished from divide operators.</simpara> |
| <simpara>Since the n4js grammar uses syntactic predicates, the content assist parser is compiled with backtracking enabled. This is always the case for Xtext’s CA parsers that rely on backtracking or predicates (local backtracking) in the production parser. This approach is both good (CA works in general) and bad (unpredictable decisions in case of error at locations prior to the cursor). Since parsing with backtracking enabled makes for a fundamental difference in how the prediction and parsing works and how the parser decides which decision paths to take, the customization patterns from the production parser are not applied 1:1 to the CA parser, but adapted instead. The content assist parser doesn’t use a freshly lexed token stream with unicode support, ASI or regular expression literals, but instead uses a synthesized token sequence which is rebuilt from the existing node model.</simpara> |
| <simpara>The token stream that is consumed by the content assist parser is therefore not created by a lexer but by the <literal>org.eclipse.n4js.ui.contentassist.NodeModelTokenSource</literal>. |
| It traverses the existing node model that is contained in the resource and was produced by the production parser. This approach has the significant advantage that any decision that was made by that parser is also immediately applicable to the content assist infrastructure. For that purpose, the leaf nodes of the node model are mapped to ANTLR token types. |
| This is achieved by the <literal>org.eclipse.n4js.ui.contentassist.ContentAssistTokenTypeMapper</literal> which is capable to provide the untyped ANTLR token type (primitive int) for a given grammar element.</simpara> |
| <simpara>Special considerations have been made for the last token in the produced source. If it overlaps with an existing leaf node but does not fully cover it, the plain Antlr lexer is used to consume the prefix that is overlapping. Since the terminals will never overlap with each other the longest match always wins without backtracking in the lexer, it is save to assume that only one token is produced from the prefix. The very last token in the <literal>org.eclipse.n4js.ui.contentassist.NodeModelTokenSource</literal> is always the EOF token (<literal>org.antlr.runtime.Token.EOF_TOKEN</literal>).</simpara> |
| <simpara>Given that the token source is equal to the prefix in the production token source, some more thought has to be put into the synthesized end of file. The production parser used the complete file to decide where to automatically insert a semicolon and where not to. This would potentially change if there was another token next to the artificial EOF. Therefore, two cases have to considered. The first one describes CA request next to an automatically inserted semicolon and the second one describes CA requests at a position where a semicolon could have been inserted if the token to the right was another one. The <literal>org.eclipse.n4js.ui.contentassist.CustomN4JSParser</literal> reflects these cases. Heuristics are applied to the end of the token sequence to decide whether a second pass has to be performed to collect yet more following elements. Based on the concrete sequence, the last automatically inserted semicolon is removed from the sequence prior to the second pass or such is a token is explicitly synthesized and appended. Besides the second pass, another special treatment is made for postfix expressions. Those may not be interrupted by a hidden semicolon so those are filtered from the resulting follow set if appropriate.</simpara> |
| <simpara>The parser is used by the <literal>org.eclipse.n4js.ui.contentassist.ContentAssistContextFactory</literal> where all relevant entry points from the super class are specialized to pass the node model in the the parser facade (<literal>org.eclipse.n4js.ui.contentassist.CustomN4JSParser</literal>). In that sense, the ContentAssistContextFactory serves as a drop-in replacement binding the default <literal>ParserBasedContentAssistContextFactory.StatefulFactory</literal>.</simpara> |
| </section> |
| </chapter> |
| <chapter xml:id="_type-system"> |
| <title>Type System</title> |
| <section xml:id="sec:Type_Model_and_Grammar" role="language-n4js"> |
| <title>Type Model and Grammar</title> |
| <simpara>The type model is used to define actual types and their relations (meta-model is defined by means of Xcore in file <literal>Types.xcore</literal>) |
| and also references to types (meta-model in <literal>TypeRefs.xcore</literal>). The type model is built via the <literal>N4JSTypesBuilder</literal> when a resource |
| is loaded and processed, and most type related tasks work only on the type model. Some types that are (internally) available |
| in N4JS are not defined in N4JS, but instead in a special, internal type language not available to N4JS developers, called N4TS |
| and defined in file <literal>Types.xtext</literal>.</simpara> |
| <simpara>The types are referenced by AST elements; vice versa the AST elements can be referenced from the types (see <literal>SyntaxRelatedTElement</literal>). |
| This backward reference is a simple reference to an EObject.</simpara> |
| <section xml:id="sec:Type_Model_Overview"> |
| <title>Type Model Overview</title> |
| <simpara>The following figure, <link linkend="fig:cd_typeAndTypeRefHierarchy">Types and Type References</link>, shows the classes of the type model and their inheritance relations, both the actual type definitions as defined in <literal>Types.xcore</literal> and the type references defined in <literal>TypeRefs.xcore</literal>. The most important type reference is the <literal>ParameterizedTypeRef</literal>; it is used for most user-defined references, for both parameterized and non-parameterized references. In the latter case, the list of type arguments is empty.</simpara> |
| <figure xml:id="fig:cd_typeAndTypeRefHierarchy"> |
| <title>Type Model Overview: Types in the upper half and Type References in the lower half.</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/06_typesystem/images/cd_typeModelHierarchy_allInOne.png"/> |
| </imageobject> |
| <textobject><phrase>cd typeModelHierarchy allInOne</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara>Most types are self-explanatory. <literal>TypeDefs</literal> is the container element used in N4TS. Note that not all types and properties of types are available in N4JS – some can only be used in the N4TS language or be inferred by the type system for internal purposes. Some types need some explanation:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><literal>TObjectPrototype</literal>: Metatype for defining built-in object types such as <literal>Object</literal> or <literal>Date</literal>, only available in N4TS.</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>VirtualBaseType</literal>: This type is not available in N4JS. It is used to define common properties provided by all types of a certain metatype. E.g., it is used for defining some properties shared by all enumerations (this was the reason for introducing this type).</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>We distinguish four kinds of types as summarized in <link linkend="tab:KindOfTypes">Kind Of Types</link>. Role is an internal construct for different kind of users who can define the special kind of type. The language column refers to the language used to specify the type; which is either N4JS or N4TS.</simpara> |
| <table xml:id="tab:KindOfTypes" frame="all" rowsep="1" colsep="1"> |
| <title>Kind of Types</title> |
| <tgroup cols="4"> |
| <colspec colname="col_1" colwidth="10*"/> |
| <colspec colname="col_2" colwidth="10*"/> |
| <colspec colname="col_3" colwidth="10*"/> |
| <colspec colname="col_4" colwidth="70*"/> |
| <thead> |
| <row> |
| <entry align="left" valign="top">Kind</entry> |
| <entry align="left" valign="top">Language</entry> |
| <entry align="left" valign="top">Role</entry> |
| <entry align="left" valign="top">Remark</entry> |
| </row> |
| </thead> |
| <tbody> |
| <row> |
| <entry align="left" valign="top"><simpara><emphasis role="strong">user</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara><emphasis role="strong">N4JS</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara><emphasis role="strong">developer</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara>User defined types, such as declared classes or functions. These types are to be explicitly defined or imported in the code.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara><emphasis role="strong">library</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara><emphasis role="strong">N4JSD</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara><emphasis role="strong">developer</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara>Type declarations only, comparable to C header files, without implementation. Used for defining the API of 3rd party libraries. These type definitions are to be explicitly defined or imported in the code.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara><emphasis role="strong">builtin</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara><emphasis role="strong">N4TS</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara><emphasis role="strong">smith</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara>Built-in ECMAScript objects interpreted as types. E.g., <literal>String</literal>, <literal>Date</literal>, <literal>Math</literal>. These types are provided by N4JS and are always available.</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara><emphasis role="strong">primitive</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara><emphasis role="strong">N4TS</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara><emphasis role="strong">smith</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara>Primitive ECMAScript (and N4JS ties), such as <literal>string</literal>, <literal>number</literal>, <literal>pathselector<T></literal>, <literal>i18n</literal>, and also <literal>any</literal>, <literal>undefined</literal> and <literal>void</literal>. These types are provided by N4JS and are always available. Primitive types are described in detail in the spec (see chapter "Primitive ECMAScript Types").</simpara></entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </table> |
| </section> |
| <section xml:id="sec:Built_in_Types"> |
| <title>Built-in and Primitive Types</title> |
| <simpara>The built-in and primitive types are not defined by the user, i.e. N4JS programmer. Instead, they are defined in special |
| internal files using the internal N4TS language: <literal>builtin_js.n4ts</literal>, <literal>builtin_n4.n4ts</literal>, <literal>primitives_js.n4ts</literal>, <literal>primitives_n4.n4ts</literal>.</simpara> |
| </section> |
| <section xml:id="sec:Type_Model_DSL" role="language-n4js"> |
| <title>Type Model DSL</title> |
| <simpara>For defining built-in types and for tests, a special DSL called N4TS is provided by means of an Xtext grammar and generated |
| tools. The syntax is similar to n4js, but of course without method or function bodies, i.e. without any statements of expressions. |
| The grammar file is found in <literal>Types.xtext</literal>.</simpara> |
| <simpara>The following list documents some differences to N4JS:</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>access modifiers directly support <literal>Internal</literal>, so no annotations are needed (nor supported) here.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>besides N4 classifiers such as classes, the following classifiers can be defined:</simpara> |
| <orderedlist numeration="loweralpha"> |
| <listitem> |
| <simpara><literal>object</literal> define classes derived from object (predefined object types) Special features:</simpara> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>indexed</simpara> |
| </entry> |
| <entry> |
| <simpara>defined what type is returned in case of index access</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </listitem> |
| <listitem> |
| <simpara><literal>virtualBase</literal> virtual base types for argument</simpara> |
| </listitem> |
| </orderedlist> |
| </listitem> |
| <listitem> |
| <simpara><literal>primitive</literal> primitive types (number, string etc.) Special features:</simpara> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>indexed</simpara> |
| </entry> |
| <entry> |
| <simpara>defined what type is returned in case of index access</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>autoboxedType</simpara> |
| </entry> |
| <entry> |
| <simpara>defines to which type the primitive can be auto boxed</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>assignmnentCompatible</simpara> |
| </entry> |
| <entry> |
| <simpara>defines to which type the primitive is assignment compatible</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </listitem> |
| <listitem> |
| <simpara>types <literal>any</literal>, <literal>null</literal>, <literal>void</literal>, <literal>undefined</literal> – special types.</simpara> |
| </listitem> |
| </orderedlist> |
| <simpara>Annotations are not supported in the types DSL.</simpara> |
| </section> |
| </section> |
| <section xml:id="sec:Type_System_Implementation" role="language-n4js"> |
| <title>Type System Implementation</title> |
| <simpara>The bulk of the type system’s functionality is implemented in packages <literal>org.eclipse.n4js.typesystem[.constraints|.utils]</literal>. |
| Client code, e.g. in validations, should only access the type system through the facade class <literal>N4JSTypeSystem</literal>. |
| Each of the main type system functions, called "judgments", are implemented in one of the concrete subclasses of |
| base class <literal>AbstractJudgment</literal>. Internally, the type system is using a constraint solver for various purposes; |
| entry point for this functionality is class <literal>InferenceContext</literal>. All these classes are a good entry point into |
| the code base, for investigating further details.</simpara> |
| <simpara>Some type information is cached (e.g., the type of an expression in the AST) and the above facade will take care |
| to read from the cache instead of re-computing the information every time, as far as possible. This type cache is |
| being filled in a dedicated phase during loading and processing of an N4JS resource; |
| see <xref linkend="sec:Type_Inference_combined_with_AST_Traversal"/> and <xref linkend="sec:N4JS_Resource_Load_States"/> for details.</simpara> |
| </section> |
| <section xml:id="sec:Type_Inference_combined_with_AST_Traversal" role="language-n4js"> |
| <title>Type Inference of AST</title> |
| <simpara>Most judgments provided by the facade <literal>N4JSTypeSystem</literal> and implemented by subclasses of <literal>AbstractJudgment</literal> are used |
| ad-hoc whenever client code requires the information they provide. This is applied, in particular, to judgments</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><literal>subtype</literal></simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>substTypeVariables</literal></simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>upperBound</literal> / <literal>lowerBound</literal></simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>For judgment <literal>type</literal> footnote:<literal role="Currently only for [language-n4js">type</literal>, not for <literal role="language-n4js">expectedType</literal>, |
| but this may be changed in a future, further refactoring.], however, the processing is very different: we make |
| sure that the entire AST, i.e. all typable nodes of the AST, will be typed up-front in a single step, which |
| takes place during the <emphasis>post-processing step</emphasis> of an N4JSResource (see <xref linkend="sec:N4JS_Resource_Load_States"/>), which |
| also has a couple other responsibilities. By triggering post-processing when client code invokes the type judgment |
| for the first time for some random AST node (at the latest; usually it is triggered earlier), we make sure that |
| this sequence is always followed.</simpara> |
| <simpara>The remainder of this section will explain this single-step typing of the entire AST in detail.</simpara> |
| <section xml:id="sec:Type_Inference_combined_with_AST_Traversal__Background"> |
| <title>Background</title> |
| <simpara>Originally, the N4JS type system could be called with any <literal>EObject</literal> at any given time, without any knowledge of the |
| context. While this looked flexible in the beginning, it caused severe problems solving some inference cases, e.g., |
| the rule environment had to be prepared from the outside and recursion problems could occur, and it was also |
| inefficient because some things had to be recalculated over and over again (although caching helped).</simpara> |
| <simpara>It is better to do type inferencing (that is, computing the type of expressions in general) in a controlled manner. |
| That is, instead of randomly computing the type of an expression in the AST, it is better to traverse the AST in a |
| well-defined traversal order. That way, it is guaranteed that certain other nodes have been visited and, if not, |
| either some special handling can kick in or an error can be reported. This could even work with XSemantics and the |
| declarative style of the rules. The difference is that by traversing the AST in a controlled manner, the rules can |
| make certain assumptions about the content of the rule-environment, such as that it always contains information |
| about type variable bindings and that it always contains information about expected types etc.</simpara> |
| <simpara>In that scenario, all AST nodes are visited and all types (and expected types) are calculated up-front. Validation |
| and other parts then do not need to actually compute types (by calling the actual, Xsemantics-generated type system); |
| instead, at that time all types have already been calculated and can simply be retrieved from the cache (this is |
| taken care of by the type system facade <literal>N4JSTypeSystem</literal>).</simpara> |
| <simpara>This also affects scoping, since all cross-references have to be resolved in this type computation step. However, |
| even for scoping this has positive effects: E.g., the receiver type in property access expressions is always visited |
| <emphasis>before</emphasis> visiting the selector. Thus it is not necessary to re-calculate the receiver type in order to perform scoping |
| for the selector.</simpara> |
| <simpara>The above refactoring was done in summer 2015. After this refactoring, we are still using Xsemantics to compute the |
| types, i.e. the <literal>type</literal> judgement in Xsemantics was largely kept as before. However, the type judgment is invoked |
| in a controlled traversal order for each typable AST node in largely one step (controlled by <literal>ASTProcessor</literal> and <literal>TypeProcessor</literal>).</simpara> |
| <simpara>The upshot of this one-step type inference is that once it is completed, the type for every typable AST node is known. |
| Instead of storing this information in a separate model, this information will be stored and persisted in the type model |
| directly, as well as in transient fields of the AST <footnote><simpara>This is not yet implemented as of September 2015; types are still stored in a separate cache, the <literal>ASTMetaInfoCache</literal>.</simpara></footnote>. Currently, this applies only to types, not expected types; |
| the inference of expected types could / should be integrated into the one-step inference as part of a future, further |
| refactoring.</simpara> |
| <table xml:id="tab:typeInferenceBeforeAfter" frame="all" rowsep="1" colsep="1"> |
| <title>Comparison of inference of type of AST nodes before / after refactoring.</title> |
| <tgroup cols="2"> |
| <colspec colname="col_1" colwidth="50*"/> |
| <colspec colname="col_2" colwidth="50*"/> |
| <thead> |
| <row> |
| <entry align="left" valign="top">Before</entry> |
| <entry align="left" valign="top">After</entry> |
| </row> |
| </thead> |
| <tbody> |
| <row> |
| <entry align="left" valign="top"><simpara>ad-hoc type inference (when client code needs the type information)</simpara></entry> |
| <entry align="left" valign="top"><simpara>up-front type inference (once for entire AST; |
| later only reading from cache)</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>started anywhere</simpara></entry> |
| <entry align="left" valign="top"><simpara>starts with root, i.e. the <literal>Script</literal></simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>Xsemantics rules traverse the AST at will, uncontrolled</simpara></entry> |
| <entry align="left" valign="top"><simpara>well-defined, controlled traversal order</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>lazy, on-demand resolution of <literal>ComputedTypeRef</literal>s (they contain the resolution logic)</simpara></entry> |
| <entry align="left" valign="top"><simpara>pro-active resolution of |
| <literal>DeferredTypeRef</literal>s (they themselves are dumb)</simpara></entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </table> |
| </section> |
| <section xml:id="sec:Triggering_Type_Inference_of_AST"> |
| <title>Triggering</title> |
| <simpara>The up-front type inference of the entire AST is part of the post-processing of every N4JSResource and is thus |
| triggered when post-processing is triggered. This happens when</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>someone directly calls <literal>#performPostProcessing()</literal> on an N4JSResource</simpara> |
| </listitem> |
| <listitem> |
| <simpara>someone directly calls <literal>#resolveAllLazyCrossReferences()</literal> on an N4JSResource,</simpara> |
| </listitem> |
| <listitem> |
| <simpara>EMF automatically resolves the first proxy, i.e. someone calls an EMF-generated getter for a value that is a proxy,</simpara> |
| </listitem> |
| <listitem> |
| <simpara>someone asks for a type for the first time, i.e. calls <literal>N4JSTypeSystem#type()</literal>,</simpara> |
| </listitem> |
| <listitem> |
| <simpara>…​</simpara> |
| </listitem> |
| </orderedlist> |
| <simpara>Usually this happens after the types builder was run with <literal>preLinking==false</literal> and before validation takes place. |
| For details, see classes <literal>PostProcessingAwareResource</literal> and <literal>N4JSPostProcessor</literal>.</simpara> |
| </section> |
| <section xml:id="sec:Traversal_Order_During_Type_Inference_of_AST"> |
| <title>Traversal Order</title> |
| <simpara>The traversal order during post-processing is a bit tricky, as some things need to be done in a top-down order (only |
| few cases, for now <footnote><simpara>In the future, the top-down order could become more important if inference of <emphasis>expected</emphasis> types is also integrated into post-processing.</simpara></footnote>), others in a bottom-up order (e.g. the main typing of AST nodes), |
| and there is a third case in which several AST nodes are processed together (constraint-based type inference).</simpara> |
| <simpara>Figure <xref linkend="fig:traversalOrder"/> provides an example of an AST and shows in which order the nodes are processed. Green |
| numbers represent top-down processing, red numbers represent bottom-up processing and blue numbers represent the |
| processing of the surrounding yellow nodes in a single step.</simpara> |
| <figure xml:id="fig:traversalOrder"> |
| <title>Order in which AST nodes are being processed during post-processing.</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/06_typesystem/images/traversalOrder.png"/> |
| </imageobject> |
| <textobject><phrase>traversalOrder</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara>In the code, this is controlled by class <literal>ASTProcessor</literal>. The two main processing methods are</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><literal>#processNode_preChildren()</literal>, which will be invoked for all AST nodes in a top-down order (so top-down processing should be put here),</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>#processNode_postChildren()</literal>, which will be invoked for all AST nodes in a bottom-up order (so bottom-up processing should be put here).</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>The common processing of groups of adjacent yellow nodes (represented in the figure by the two yellow/brown |
| triangles) is achieved by <literal>PolyProcessor</literal> telling the <literal>TypeProcessor</literal> to</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>ignore certain nodes (all yellow nodes) and</simpara> |
| </listitem> |
| <listitem> |
| <simpara>invoke method <literal>PolyProcessor#inferType()</literal> for the root yellow node in each group (only the root!). |
| . |
| For details, see the two methods <literal>#isResponsibleFor()</literal> and <literal>#isEntryPoint()</literal> in <literal>PolyProcessor</literal>.</simpara> |
| </listitem> |
| </orderedlist> |
| </section> |
| <section xml:id="sec:Cross_References_During_Type_Inference_of_AST"> |
| <title>Cross-References</title> |
| <simpara>While typing the entire AST, cross-references need special care. Three cases of cross-references need to be distinguished:</simpara> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>backward reference</simpara> |
| </entry> |
| <entry> |
| <simpara>= cross-reference within the same file to an AST node that was already processed</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>always legal</simpara> |
| </listitem> |
| <listitem> |
| <simpara>processing: simply read the type from the cache that is currently being filled</simpara> |
| </listitem> |
| </itemizedlist> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>forward reference</simpara> |
| </entry> |
| <entry> |
| <simpara>= cross-reference within the same file to an AST node that was not yet processed</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>usually illegal<?asciidoc-br?> |
| exception: legal if reference points to an <emphasis>identifiable subtree</emphasis> (a subtree of an AST with an identifiable element at its root)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>processing: forward process the identifiable subtree and report back the type of its root</simpara> |
| </listitem> |
| </itemizedlist> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>references to other files</simpara> |
| </entry> |
| <entry> |
| <itemizedlist> |
| <listitem> |
| <simpara>always legal, because they can, by nature, only point to an identifiable subtree in the other file’s AST</simpara> |
| </listitem> |
| <listitem> |
| <simpara>processing: either …​</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>read type from TModule obtained from index (if available), or</simpara> |
| </listitem> |
| <listitem> |
| <simpara>load other file from source, trigger its post-processing (if not in progress or completed already), forward process the identifiable subtree (if not processed already) and report back the type of its root.</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| </itemizedlist> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| <simpara>Note that for references to an ancestor (upward references) or successor (downward references) within an AST, the |
| classification as a forward or backward reference depends on whether we are in top-down or bottom-up processing. |
| Figure <link linkend="fig:upwardDownward">Upward Downward</link> illustrates this: the left and right side show the same AST but on the |
| left side we assume a top down processing whereas on the right we assume a bottom up processing. On both sides, |
| backward references are shown in green ink (because they are unproblematic and always legal) and forward references |
| are shown in red ink. Now, looking at the two arrows pointing from a node to its parent, we see that it is classified |
| as a backward reference on the left side (i.e. top down case) but as a forward reference on the right side (i.e. bottom |
| down case). Conversely, an arrow from a node to its child is classified as a forward reference on the left side and as |
| a backward reference on the right side. Arrows across subtrees, however, are classified in the same way on the left and |
| right side (see the horizontal arrows at the bottom).</simpara> |
| <figure xml:id="fig:upwardDownward"> |
| <title>Backward and forward references in top-down and bottom-up processing.</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/06_typesystem/images/upwardDownward.png"/> |
| </imageobject> |
| <textobject><phrase>upwardDownward</phrase></textobject> |
| </mediaobject> |
| </figure> |
| </section> |
| <section xml:id="sec:Function_Accessor_Bodies_During_Type_Inference_of_AST"> |
| <title>Function/Accessor Bodies</title> |
| <simpara>An important exception to the basic traversal order shown in Figure <xref linkend="fig:traversalOrder"/> is that the body of all |
| functions (including methods) and field accessors is postponed until the end of processing. This is used to avoid |
| unnecessary cycles during type inference due to a function’s body making use of the function itself or some other |
| declarations on the same level as the containing function. For example, the following code relies on this:</simpara> |
| <programlisting language="n4js" linenumbering="unnumbered">let x = f(); |
| function f(): X { |
| if(x) { |
| // XPECT noerrors --> "any is not a subtype of X." at "x" |
| return x; |
| } |
| return new X(); |
| }</programlisting> |
| <simpara>Similar situation using fields and methods:</simpara> |
| <programlisting language="n4js" linenumbering="unnumbered">class C { |
| d = new D(); |
| mc() { |
| // XPECT noerrors --> "any is not a subtype of D." at "this.d" |
| let tmp: D = this.d; |
| } |
| } |
| class D { |
| md() { |
| new C().mc(); |
| } |
| }</programlisting> |
| <simpara>For details of this special handling of function bodies, see method <literal>ASTProcessor#isPostponedNode(EObject)</literal> and field |
| <literal>ASTMetaInfoCache#postponedSubTrees</literal> and the code using it. For further investigation, change <literal>isPostponedNode()</literal> to always |
| return false and debug with the two examples above (which will then show the incorrect errors mentioned in the XPECT |
| comments) or run tests to find more cases that require this handling.</simpara> |
| </section> |
| <section xml:id="sec:Poly_Expressions_During_Type_Inference_of_AST"> |
| <title>Poly Expressions</title> |
| <simpara>Polymorphic expressions, or <emphasis>poly expressions</emphasis> for short, are expressions for which the actual type depends on the |
| expected type and/or the expected type depends on the actual type. They require constraint-based type inference |
| because the dependency between the actual and expected type can introduce dependency cycles between the types of |
| several AST nodes which are best broken up by using a constraint-based approach. This is particularly true when |
| several poly expressions are nested. Therefore, poly expressions are inferred neither in top-down nor in bottom-up |
| order, but all together by solving a single constraint system.</simpara> |
| <simpara>Only a few types of expressions can be polymorphic; they are called <emphasis>poly candidates</emphasis>: array literals, object literals, |
| call expressions, and function expressions. The following rules tell whether a poly candidate is actually poly:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><literal>ArrayLiteral</literal> — always poly (because their type cannot be declared explicitly).</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>ObjectLiteral</literal> — if one or more properties do not have a declared type.</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>CallExpression</literal> — if generic & not parameterized.</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>FunctionExpression</literal> — if return type or type of one or more formal parameters is undeclared.</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>This is a simplified overview of these rules, for details see method <literal>#isPoly(Expression)</literal> in <literal>AbstractPolyProcessor</literal>.</simpara> |
| <simpara>The main logic for inferring the type of poly expressions is found in method <literal>#inferType()</literal> in class <literal>PolyProcessor</literal>. |
| It is important to note that this method will only be called for root poly expressions (see above). In short, the basic |
| approach is to create a new, empty <literal>InferenceContext</literal>, i.e. constraint system, add inference variables and constraints for |
| the root poly expression and all its nested poly expressions, solve the constraint system and use the types in the solution |
| as the types of the root and nested poly expressions. For more details see method <literal>#inferType()</literal> in class <literal>PolyProcessor</literal>.</simpara> |
| <simpara>So, this means that nested poly expressions do not introduce a new constraint system but instead simply extend their parent |
| poly’s constraint system by adding additional inference variables and constraints. <emphasis role="strong">But not every nested expression that is |
| poly is a nested poly expression in that sense!</emphasis> Sometimes, a new constraint system has to be introduced. For example:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>child poly expressions that appear as argument to a call expression are nested poly expressions (i.e. inferred in same constraint system as the parent call expression),</simpara> |
| </listitem> |
| <listitem> |
| <simpara>child poly expressions that appear as target of a call expression are <emphasis role="strong">not</emphasis> nested poly expressions and a new constraint system has to be introduced for them.</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>For details see method <literal>#isRootPoly()</literal> in <literal>AbstractPolyProcessor</literal> and its clients.</simpara> |
| </section> |
| <section xml:id="sec:Constraint_Solver_used_During_Type_Inference_of_AST"> |
| <title>Constraint Solver</title> |
| <simpara>The simple constraint solver used by the N4JS type system, mainly for the inference of poly expressions, is implemented |
| by class <literal>InferenceContext</literal> and the other classes in package <literal>org.eclipse.n4js.typesystem.constraints</literal>.</simpara> |
| <simpara>The constraint solving algorithm used here is largely modeled after the one defined in <literal>The Java Language Specification 8</literal>, |
| Chapter 18, but was adjusted in a number of ways, esp. by removing functionality not required for N4JS (e.g. primitive types, |
| method overloading) and adding support for specific N4JS language features (e.g. union types, structural typing).</simpara> |
| <simpara>For details see the API documentation of class <literal>InferenceContext</literal>.</simpara> |
| </section> |
| <section xml:id="sec:Type_Guards_During_Type_Inference_of_AST"> |
| <title>Type Guards</title> |
| <simpara>During AST post-processing, the control and data flow analyses are performed. |
| This means, that first a flow graph is created and then traversed. |
| During the traversal, a type guard analysis is performed which saves information by evaluating <literal>instanceof</literal> expressions. |
| As a result, the analysis provides a reliable set of RHS expressions of <literal>instanceof</literal> expressions for each AST element of type <literal>IdentifierRef</literal>.</simpara> |
| <simpara>This set is evaluated in the <literal>TypeJudgments.java</literal> when typing <literal>IdentifierRef</literal> elements. |
| In case the set is not empty, the types of all elements is calculated. |
| The type of the <literal>IdentifierRef</literal> will then become the intersection of its original type and all types previously calculated.</simpara> |
| </section> |
| </section> |
| <section xml:id="sec:Structural_Typing"> |
| <title>Structural Typing</title> |
| <simpara>Structural typing as an optional subtyping mode in N4JS is implemented in <literal>StructuralTypingComputer</literal>, activated depending on |
| the value of property <literal>typingStrategy</literal> in <literal>ParameterizedTypeRef</literal> and its subclasses.</simpara> |
| </section> |
| </chapter> |
| <chapter xml:id="_type-index"> |
| <title>Type Index</title> |
| <section xml:id="sec:Type_Index_Design_Rationale" role="language-n4js"> |
| <title>Design Rationale</title> |
| <simpara>We use a separate types model to represent types, see <xref linkend="sec:Type_Model_and_Grammar"/>. Declared elements (e.g., classes) |
| in N4JS are parsed and a new types model instance is derived from them. All type references (of the N4JS <link linkend="AC">AST</link>) |
| are then bound to these type instances and not to the N4JS declaration. However, there exists a relation between a type |
| and its declaration. The type instances (which are EObjects) are added to the resource of the N4JS file as part of |
| the public interface of the resource. This public interface is represented by a <literal>TModule</literal>. While the actual source code |
| is the first element of a resource (index 0), the module is stored at index 1. It contains the derived type information, |
| the information about exported variables and functions as well as information about the project and vendor. The Xtext |
| serializer ignores the additional element. Besides, the complete type instances are stored in the user data section of |
| the <literal>IEObjectDescription</literal> of the <literal>TModule</literal>. Since the user data only allows strings to be stored, the EObjects are serialized |
| (within a virtual resource). When a reference is then bound to a type, the type can be directly recreated (deserialized) |
| from the user data. The deserialized EObject is then added to the appropriate resource. It is not necessary to load the |
| complete file just to refer to a type from that file.</simpara> |
| <simpara>The design relies on two key features of Xtext:</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>Besides the parsed text (i.e., the AST), other elements can be stored in a resource, which are then ignored by |
| the Xtext serializer, while still being properly contained in an EMF resource.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>The <literal>DerivedStateAwareResource</literal> allows some kind of post processing steps when reading a resource. This enables a custom |
| class, here <literal>N4JSDerivedStateComputer</literal>, to create the types models (TClass, TRole and so on) from the parsed <literal>N4ClassDeclaration</literal>, |
| <literal>N4RoleDeclaration</literal> and so on.</simpara> |
| </listitem> |
| </orderedlist> |
| <section xml:id="sec:Getting_the_Xtext_Index_Content_IResourceDescriptions"> |
| <title>Getting the Xtext Index (<literal>IResourceDescriptions</literal>) Content</title> |
| <simpara>An instance of the <literal>IResourceDescriptions</literal> can be acquired from the resource description provider. Just like all services |
| in Xtext, this can be injected into the client code as well. The resource descriptions accepts a non-null resource set. |
| The resource set argument is mandatory to provide the index with the proper state. We are differentiating between three |
| different state. The first one is the persisted one, basically the builder state is a resource description as well, and |
| it provides a content that is based on the persisted state of the files (in our case the modules and package.json file) |
| on the file system. The second one is the live scoped index, this is modification and dirty state aware. Namely when using |
| this resource descriptions then an object description will be searched in the resource set itself first, then in the dirty |
| editor’s index, finally among the persisted ones. The third index is the named builder scoped one. This index should not be |
| used by common client code, since it is designed and implemented for the Xtex builder itself.</simpara> |
| <simpara>A resource set and hence the index can be acquired from the N4JS core, in such cases an optional N4JS project can be specified. |
| The N4JS project argument is used to retrieve the underlying Eclipse resource project (if present) and get the resource set from |
| the resource set provider. This is completely ignored when the application is running in headless mode and Eclipse resource |
| projects are not available. It is also important to note that the resource set is always configured to load only the persisted |
| states.</simpara> |
| <simpara>When the Eclipse platform is running, the workspace is available and the all N4JS projects are backed by an Eclipse resource |
| project. With the Eclipse resource project the resource sets can be initialized properly via the resource set initializer |
| implementations. This mechanism is used to get the global objects (such as console) and the built-in types (such as string, |
| number) into the resource set via the corresponding resource set adapters. In the headless case a special resource set |
| implementation is used; <literal>ResourceSetWithBuiltInScheme</literal>. This implementation is responsible to initialize the globals and the |
| built-in types into itself.</simpara> |
| </section> |
| </section> |
| <section xml:id="sec:Design_Overview" role="language-n4js"> |
| <title>Design Overview</title> |
| <simpara><link linkend="fig:cd_TypeModelWithXtextIndex">Type Model With Xtext Index</link> shows a simplified UML class diagram with the involved |
| classes. In the figure, a class (defined as N4ClassExpression in the <link linkend="AC">AST</link> and its type TClass) is used as a sample, |
| declared type—roles or enums are handled similarly.</simpara> |
| <simpara>In the Eclipse project build the <literal>N4JSResourceDescriptionManager</literal> (resp. by the logic of its super class) is called by the |
| <literal>N4JSGenerateImmediatelyBuilderState</literal> to get the resource description for a resource. The resource description manager loads |
| the resource to create / update the resource descriptions. Loading an Xtext resource means that it is reparsed again. |
| All cross references are handled here only by the lazy linker so that the node model will contain an unresolved proxy |
| for all cross references.</simpara> |
| <simpara>After the resource is loaded there is a derived state installed to the resource. For this the <literal>N4JSDerivedStateComputer</literal> will |
| be called. It will take the parse result (= EObject tree in first slot of the resource) and navigate through these objects |
| to create type trees for each encountered exportable object that are stored in exported <literal>TModule</literal> of the resource. |
| <link linkend="fig:cd_CreateTypeFromAST">Create Type From AST</link>, a snippet of <link linkend="fig:cd_TypeModelWithXtextIndex">Type Model with Xtext Index</link>, |
| shows only the classes involved when creating the types from the resource.</simpara> |
| <figure xml:id="fig:cd_CreateTypeFromAST" role="center"> |
| <title>Type From AST</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/07_typeindex/images/cd_CreateTypeFromAST.png"/> |
| </imageobject> |
| <textobject><phrase>cd CreateTypeFromAST</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <figure xml:id="fig:cd_TypeModelWithXtextIndex" role="center"> |
| <title>Create Type From Index</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/07_typeindex/images/cd_CreateTypeFromIndex.png"/> |
| </imageobject> |
| <textobject><phrase>cd CreateTypeFromIndex</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <figure xml:id="fig:cd_SerializeToIndex" role="center"> |
| <title>Serialize To Index</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/07_typeindex/images/cd_SerializeToIndex.png"/> |
| </imageobject> |
| <textobject><phrase>cd SerializeToIndex</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara role="language-javascript">For these elements types have to be derived as they are exportable: <literal>N4ClassDeclaration</literal>, <literal>N4RoleDeclaration</literal>, <literal>N4InterfaceDeclaration</literal>, |
| <literal>N4EnumDeclaration</literal>, <literal>ExportedVariableDeclaration</literal> and <literal>FunctionDeclaration</literal>.</simpara> |
| <simpara>After loading and initializing the resources now all cross references in the resources are resolved. For this the |
| <literal>ErrorAwareLinkingService</literal> is used. This class will in turn call the <literal>N4JSScopeProvider</literal> to first try to do scoping locally |
| but eventually also delegate to the global scope provider to find linked elements outside the current resource. This |
| will be done e.g. for every import statement inside the N4JS resource.</simpara> |
| <simpara>For determine the global scope all visible containers for this resource are calculated. For this the project description |
| (= loaded package.json file) is used to determine which folders of the current project should be included for looking for |
| N4JS resources. Also all referenced projects and their resources are added to the visible containers. For these containers |
| <literal>N4JSGlobalScopeProvider</literal> builds up a container scope. This container scope will be a <literal>N4JSTypesScope</literal> instance.</simpara> |
| <simpara>For the actual linked element in the resource to be resolved, its fully qualified name is used. This name is calculated by |
| using the <literal>IQualifiedNameConverter</literal>. We bound a custom class named <literal>N4JSQualifiedNameConverter</literal> who converts the <literal>/</literal> inside the |
| qualified name to a dot, so e.g. <literal>my/module/MyFileName</literal> is converted to <literal>my.module.MyFileName</literal>. Btw. the initial qualified name |
| was derived from the node model.</simpara> |
| <simpara>With this qualified name <literal>N4JSTypeScope.getSingleElement</literal> is called. This method does the actual resolving of the cross reference. |
| For this the URI of the cross reference is used to determine the linked resource.</simpara> |
| <simpara>There are now three cases:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>If the resource which contains the linked EObject is already loaded the EObject description found for the URI is returned</simpara> |
| </listitem> |
| <listitem> |
| <simpara>If the resource is not loaded but the first slot of the resource is empty the referenced type is tried to be rebuild from |
| an existing resource description for the linked resource inside the Xtext index.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>If the resource is not loaded and the first slot is set, the linked EObject will be resolved with the fragment of the |
| given URI.</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>While calculating the resource description for a <literal>N4JSResource</literal>, the EObject descriptions of their exported objects have to be |
| calculated as well. For this the <literal>N4JSResourceDescriptionStrategy</literal> is used. For computing the exported objects of a resource only |
| the root <literal>TModule</literal> and its contained types and variables are taken in consideration.</simpara> |
| <simpara>The EObjectDescriptions for a n4js resource include:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>An exported representation of the derived <literal>TModule</literal>. This carries these properties:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>a qualified name (e.g. <literal>my.module.MyFileName</literal> when the resource is stored under <literal>src/my/module/MyFileName.js</literal> in the project and |
| the project description has marked src has src folder). The calculation of the qualified name is delegated to the <literal>N4JSNamingUtil</literal>.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>the user data which is the serialized string of the exported <literal>TModule</literal> itself. It includes the types determined for this |
| resource, so for every element found in this resource, that is contained in an <literal>ExportStatement</literal>, an EObject has been created |
| before in <literal>N4JSDerivedStateComputer</literal>. In most cases this an EObject extending <literal>Type</literal> from the types model, e.g. <literal>TClass</literal> for |
| <literal>N4ClassDeclaration</literal>. There is an exception for <literal>ExportedVariableDeclaration</literal> where <literal>TVariable</literal> is used as representative (and this |
| EObject is not contained in the types model only in the N4JS model). For usability reasons (quicker quick fix etc.), also |
| top level types not exported yet are stored in the <literal>TModel</literal>.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>the information on project and vendor id are part of the module descriptor.</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara>Descriptions for all top level types that are defined in the resource. These descriptions do not have any special properties, |
| so they just have a name.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>All exported variables are also described in the resource description. They don’t carry any special information either.</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>The EObjectDescription for an EObject contained in an <literal>ExportStatement</literal>:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>the qualified name of the module export (e.g. for a <literal>N4ClassDeclaration</literal> the qualified name <literal>my.module.MyFileName.MyClassName</literal> would |
| be produced, when the resource is stored under <literal>src/my/module/MyFileName.js</literal> in the project, the project description has marked |
| src has src folder and the N4 class uses the name MyClassName]). The calculation of the qualified name is delegated to the |
| <literal>N4JSNamingUtil</literal>.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>the EObject represented by the EObject description, here this is not the actual EObject from N4JS but the type EObject from |
| the TypeSystem, that has been inferenced by using <literal>N4JSTypeInferencer</literal></simpara> |
| </listitem> |
| <listitem> |
| <simpara>the user data is only an empty map for this EObjectDescription</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>With this the resource description for a resource should be fully created / updated. <link linkend="fig:cd_SerializeToIndex">Serialize to Index</link> |
| shows the classes involved creating the resource and EObjectDescriptions, along with the serialized type information.</simpara> |
| </section> |
| <section xml:id="sec:N4JS_Resource_Load_States"> |
| <title>N4JS Resource Load States</title> |
| <simpara>Below state diagram depicts the state transitions when loading and resolving an N4JS resource.</simpara> |
| <figure role="center"> |
| <title>N4JS Resource resolution states</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/07_typeindex/images/resource_set_load_states.png"/> |
| </imageobject> |
| <textobject><phrase>resource set load states</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara>Additionally, the following table relates the values of the resource’s flags to the states.</simpara> |
| <informaltable frame="all" rowsep="1" colsep="1"> |
| <tgroup cols="9"> |
| <colspec colname="col_1" colwidth="11.1111*"/> |
| <colspec colname="col_2" colwidth="11.1111*"/> |
| <colspec colname="col_3" colwidth="11.1111*"/> |
| <colspec colname="col_4" colwidth="11.1111*"/> |
| <colspec colname="col_5" colwidth="11.1111*"/> |
| <colspec colname="col_6" colwidth="11.1111*"/> |
| <colspec colname="col_7" colwidth="11.1111*"/> |
| <colspec colname="col_8" colwidth="11.1111*"/> |
| <colspec colname="col_9" colwidth="11.1112*"/> |
| <thead> |
| <row> |
| <entry align="left" valign="top">State</entry> |
| <entry align="left" valign="top">Parse Result</entry> |
| <entry align="left" valign="top">AST</entry> |
| <entry align="left" valign="top">TModule</entry> |
| <entry align="left" valign="top">ASTMetaInfoCache</entry> |
| <entry align="left" valign="top">loaded</entry> |
| <entry align="left" valign="top">fullyInitialized</entry> |
| <entry align="left" valign="top">fullyProcessed</entry> |
| <entry align="left" valign="top">reconciled</entry> |
| </row> |
| </thead> |
| <tbody> |
| <row> |
| <entry align="left" valign="top"><simpara><emphasis role="strong">Created</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara><literal>null</literal></simpara></entry> |
| <entry align="left" valign="top"><simpara><literal>null</literal></simpara></entry> |
| <entry align="left" valign="top"><simpara><literal>null</literal></simpara></entry> |
| <entry align="left" valign="top"><simpara><literal>null</literal></simpara></entry> |
| <entry align="left" valign="top"><simpara>false</simpara></entry> |
| <entry align="left" valign="top"><simpara>false</simpara></entry> |
| <entry align="left" valign="top"><simpara>false</simpara></entry> |
| <entry align="left" valign="top"><simpara>false</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara><emphasis role="strong">Created'</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara><literal>null</literal></simpara></entry> |
| <entry align="left" valign="top"><simpara><literal>null</literal></simpara></entry> |
| <entry align="left" valign="top"><simpara><literal>null</literal></simpara></entry> |
| <entry align="left" valign="top"><simpara><literal>null</literal></simpara></entry> |
| <entry align="left" valign="top"><simpara>false</simpara></entry> |
| <entry align="left" valign="top"><simpara>true</simpara></entry> |
| <entry align="left" valign="top"><simpara>false</simpara></entry> |
| <entry align="left" valign="top"><simpara>false</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara><emphasis role="strong">Loaded</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara>available</simpara></entry> |
| <entry align="left" valign="top"><simpara>with lazy linking proxies</simpara></entry> |
| <entry align="left" valign="top"><simpara><literal>null</literal></simpara></entry> |
| <entry align="left" valign="top"><simpara><literal>null</literal></simpara></entry> |
| <entry align="left" valign="top"><simpara>true</simpara></entry> |
| <entry align="left" valign="top"><simpara>false</simpara></entry> |
| <entry align="left" valign="top"><simpara>false</simpara></entry> |
| <entry align="left" valign="top"><simpara>false</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara><emphasis role="strong">Pre-linked</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara>available</simpara></entry> |
| <entry align="left" valign="top"><simpara>with lazy linking proxies</simpara></entry> |
| <entry align="left" valign="top"><simpara>with stubs</simpara></entry> |
| <entry align="left" valign="top"><simpara><literal>null</literal></simpara></entry> |
| <entry align="left" valign="top"><simpara>true</simpara></entry> |
| <entry align="left" valign="top"><simpara>true</simpara></entry> |
| <entry align="left" valign="top"><simpara>false</simpara></entry> |
| <entry align="left" valign="top"><simpara>false</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara><emphasis role="strong">Fully Initialized</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara>available</simpara></entry> |
| <entry align="left" valign="top"><simpara>with lazy linking proxies</simpara></entry> |
| <entry align="left" valign="top"><simpara>with DeferredTypeRefs</simpara></entry> |
| <entry align="left" valign="top"><simpara><literal>null</literal></simpara></entry> |
| <entry align="left" valign="top"><simpara>true</simpara></entry> |
| <entry align="left" valign="top"><simpara>true</simpara></entry> |
| <entry align="left" valign="top"><simpara>false</simpara></entry> |
| <entry align="left" valign="top"><simpara>false</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara><emphasis role="strong">Fully Processed</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara>available</simpara></entry> |
| <entry align="left" valign="top"><simpara>available</simpara></entry> |
| <entry align="left" valign="top"><simpara>available</simpara></entry> |
| <entry align="left" valign="top"><simpara>available</simpara></entry> |
| <entry align="left" valign="top"><simpara>true</simpara></entry> |
| <entry align="left" valign="top"><simpara>true</simpara></entry> |
| <entry align="left" valign="top"><simpara>true</simpara></entry> |
| <entry align="left" valign="top"><simpara>false</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara><emphasis role="strong">Loaded from Description</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara><literal>null</literal></simpara></entry> |
| <entry align="left" valign="top"><simpara>proxy</simpara></entry> |
| <entry align="left" valign="top"><simpara>available</simpara></entry> |
| <entry align="left" valign="top"><simpara><literal>null</literal></simpara></entry> |
| <entry align="left" valign="top"><simpara>indeterminate</simpara></entry> |
| <entry align="left" valign="top"><simpara>true</simpara></entry> |
| <entry align="left" valign="top"><simpara>true</simpara></entry> |
| <entry align="left" valign="top"><simpara>false</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara><emphasis role="strong">Loaded from Description'</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara><literal>null</literal></simpara></entry> |
| <entry align="left" valign="top"><simpara>proxy</simpara></entry> |
| <entry align="left" valign="top"><simpara>with DeferredTypeRefs</simpara></entry> |
| <entry align="left" valign="top"><simpara><literal>null</literal></simpara></entry> |
| <entry align="left" valign="top"><simpara>indeterminate</simpara></entry> |
| <entry align="left" valign="top"><simpara>true</simpara></entry> |
| <entry align="left" valign="top"><simpara>true</simpara></entry> |
| <entry align="left" valign="top"><simpara>false</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara><emphasis role="strong">Fully Initialized ®</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara>available</simpara></entry> |
| <entry align="left" valign="top"><simpara>with lazy linking proxies</simpara></entry> |
| <entry align="left" valign="top"><simpara>available</simpara></entry> |
| <entry align="left" valign="top"><simpara><literal>null</literal></simpara></entry> |
| <entry align="left" valign="top"><simpara>indeterminate</simpara></entry> |
| <entry align="left" valign="top"><simpara>true</simpara></entry> |
| <entry align="left" valign="top"><simpara>false</simpara></entry> |
| <entry align="left" valign="top"><simpara>true</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara><emphasis role="strong">Fully Processed ®</emphasis></simpara></entry> |
| <entry align="left" valign="top"><simpara>available</simpara></entry> |
| <entry align="left" valign="top"><simpara>available</simpara></entry> |
| <entry align="left" valign="top"><simpara>available</simpara></entry> |
| <entry align="left" valign="top"><simpara>available</simpara></entry> |
| <entry align="left" valign="top"><simpara>indeterminate</simpara></entry> |
| <entry align="left" valign="top"><simpara>true</simpara></entry> |
| <entry align="left" valign="top"><simpara>true</simpara></entry> |
| <entry align="left" valign="top"><simpara>true</simpara></entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| <simpara>Remarks:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>oddities are shown in red ink, in the above figure.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>in the above figure:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>"AST (proxy)" means the AST consists of only a single node of type <literal>Script</literal> and that is a proxy,</simpara> |
| </listitem> |
| <listitem> |
| <simpara>"AST (lazy)" means the AST is completely created, but cross-references are represented by unresolved |
| Xtext lazy-linking proxies,</simpara> |
| </listitem> |
| <listitem> |
| <simpara>"TModule (stubs)" means the TModule has been created with incomplete information, e.g. return types of |
| all TMethods/TFunctions will be <literal>null</literal> (only used internally by the incremental builder),</simpara> |
| </listitem> |
| <listitem> |
| <simpara>"TModule (some deferred)" means the TModule has been created, does not contain stub, but some |
| `TypeRef`s are `DeferredTypeRef`s that are supposed to be replaced by proper `TypeRef`s during post-processing.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>"AST" and "TModule" means the AST/TModule is available without any qualifications.</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara>state <emphasis role="strong">Created'</emphasis>: only required because Xtext does not clear flag <literal>fullyInitialized</literal> upon unload; that is done lazily |
| when <literal>#load()</literal> is invoked at a later time.. Thus, we do not reach state <emphasis role="strong">Created</emphasis> when unloading from state |
| <emphasis role="strong">Fully Initialized</emphasis> but instead get to state <emphasis role="strong">Created'</emphasis>. To reach state <emphasis role="strong">Created</emphasis> from <emphasis role="strong">Fully Initialized</emphasis> we have to |
| explicitly invoke <literal>#discardDerivedState()</literal> before(!) unloading.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>state <emphasis role="strong">Loaded from Description'</emphasis>: transition <literal>#unloadAST()</literal> from state <emphasis role="strong">Fully Initialized</emphasis> leaks a non-post-processed |
| TModule into state <emphasis role="strong">Loaded from Description</emphasis>, which is inconsistent with actually loading a TModule from the index, |
| because those are always fully processed. Hence, the addition of state <emphasis role="strong">Loaded from Description'</emphasis>.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>states <emphasis role="strong">Fully Initialized ®</emphasis> and <emphasis role="strong">Fully Processed ®</emphasis>: these states are reached via reconciliation of a pre-existing |
| TModule with a newly loaded AST. These states differ in an unspecified way from their corresponding non-reconciled |
| states. For example, in state <emphasis role="strong">Fully Initialized ®</emphasis> the TModule does not contain any DeferredTypeRefs while, at the |
| same time, the TModule isn’t fully processed, because proxy resolution, typing, etc. have not taken place, yet.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>TODO old text (clarify this; I could not reproduce this behavior): when <literal>unloadAST</literal> is called, <literal>fullyInitialized</literal> remains |
| unchanged. This is why the value of <literal>fullyInitialized</literal> should be indeterminate in row <emphasis role="strong">Loaded from Description</emphasis>; it |
| depends on the previous value if the state <emphasis role="strong">Loaded from Description</emphasis> was reached by calling <literal>unloadAST</literal>.</simpara> |
| </listitem> |
| </itemizedlist> |
| </section> |
| <section xml:id="sec:Type_Builder" role="language-n4js"> |
| <title>Types Builder</title> |
| <simpara>When a resource is loaded, it is parsed, linked, post-processed, validated and eventually compiled. For linking and validation |
| type information is needed, and as described above the type information is created automatically when loading a resource using |
| the types builder. <link linkend="fig:ad_resourceLoading">Resource Loading</link> shows an activity model with the different actions performed when |
| a resource is loaded.</simpara> |
| <figure xml:id="fig:ad_resourceLoading" role="center"> |
| <title>Activity Diagram, Resource Loading</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/07_typeindex/images/ad_resourceLoading.png"/> |
| </imageobject> |
| <textobject><phrase>ad resourceLoading</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara>The blue colored steps are standard Xtext workflow. Handling the TModule and storing that in the index are N4 specific (red background).</simpara> |
| <section xml:id="sec:Type_Inference_not_allowed_in_Type_Builder"> |
| <title>Type Inference not allowed in Types Builder</title> |
| <simpara>A crucial point in the workflow described above is the combination of types model building and type inference. In some cases, |
| the type of a given element is not directly stated in the AST but has to be inferred from an expression and other types. For |
| example, when a variable declaration does not declare the variable’s type explicitly but provides an initializer expression, |
| the actual type of the variable is inferred to be the type of the expression.</simpara> |
| <simpara>However, the types builder cannot be allowed to use type inference, mainly for two reasons:</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>type inference through Xsemantics could lead to resolution of cross-references (i.e. EMF proxies generated by the lazy |
| linker) and because the types builder is triggered when getContents() is called on the containing <literal>N4JSResource</literal> this would |
| break a basic contract of EMF resources.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>type inference could cause other resources to be loaded which would lead to problems (infinite loops or strange results) |
| in case of circular dependencies. This is illustrated in <link linkend="fig:sd_typesBuilder_problem">Types Builder Problem</link> and |
| <link linkend="fig:sd_typesBuilder_proxies">Types Builder Proxies</link>.</simpara> |
| </listitem> |
| </orderedlist> |
| <simpara>Therefore, whenever the type of a particular element has to be inferred, the types builder will use a special type reference |
| called <literal>DeferredTypeRef</literal> <footnote><simpara>The <literal role="language-n4js">DeferredTypeRef</literal> has replaced the old <literal role="language-n4js">ComputedTypeRef</literal> that had been used until Summer 2015; those were resolved lazily when the type was actually needed (triggered on demand). For a discussion of this change see <xref linkend="sec:Type_Inference_combined_with_AST_Traversal__Background"/> and in particular <xref linkend="tab:typeInferenceBeforeAfter"/>.</simpara></footnote>, |
| in order to defer the actual type inference to a later stage, i.e. the post-processing stage.</simpara> |
| <figure xml:id="fig:sd_typesBuilder_problem" role="center"> |
| <title>Sequence Diagram, Types Builder Problem</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/07_typeindex/images/sd_typesBuilder_problem.png"/> |
| </imageobject> |
| <textobject><phrase>sd typesBuilder problem</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <figure xml:id="fig:sd_typesBuilder_proxies" role="center"> |
| <title>Sequence Diagram, Types Builder with Proxies</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/07_typeindex/images/sd_typesBuilder_proxies.png"/> |
| </imageobject> |
| <textobject><phrase>sd typesBuilder proxies</phrase></textobject> |
| </mediaobject> |
| </figure> |
| </section> |
| <section xml:id="sec:ComputedTypeReferences"> |
| <title>Deferred Type References</title> |
| <simpara>Whenever type inference would be required to obtain the actual type of an element, the types builder will insert a stub to defer |
| actual type inference (see previous section). A dedicated subclass of <literal>TypeRef</literal>, called <literal>DeferredTypeRef</literal>, is used that contains neither |
| the actual type information nor any information necessary to perform the type inference at a later point in time. Later, this |
| <literal>DeferredTypeRef</literal> will be replaced during post-processing, see <literal>TypeDeferredProcessor</literal>.</simpara> |
| <simpara>All <literal>DeferredTypeRef</literal>s will be replaced by the actual types during post-processing. One important reason for resolving |
| all <literal>DeferredTypeRef</literal>s as early as possible is that they are not suited for serialization and therefore have to be removed |
| from the types model before populating the Xtext index, which includes serializing the TModule into the user data of the |
| root element. This is always assured by the logic that manages the triggering of the post-processing phase.</simpara> |
| <simpara>To manually trigger resolution of all <literal>DeferredTypeRef</literal>s in a given types model, simply call method <literal>performPostProcessing(CancelIndicator)</literal> |
| of the containing <literal>N4JSResource</literal> (should never be required by client code such as validations).</simpara> |
| </section> |
| <section xml:id="sec:Use_cases_of_ComputedTypeRef"> |
| <title>Use cases of DeferredTypeRef</title> |
| <simpara>Currently, <literal>DeferredTypeRef</literal>s are created by the types builder only in these cases:</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>actual type of an exported TVariable if no declared type but an initialization expression are given.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>actual type of a TField if no declared type but an initialization expression are given.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>actual type of properties of ObjectLiterals if not declared explicitly.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>actual type of formal parameters and return value of function expressions if not declared explicitly.</simpara> |
| </listitem> |
| </orderedlist> |
| <simpara>Note that this overview might easily get out-dated; see references to class <literal>DeferredTypeRef</literal> in the code.</simpara> |
| </section> |
| </section> |
| <section xml:id="sec:Incremental_Builder_Overview" role="language-n4js"> |
| <title>Incremental Builder (Overview)</title> |
| <simpara>This section provides a brief overview of how the incremental builder works.</simpara> |
| <simpara>General remarks:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>The N4JS incremental builder is a combination of Eclipse builder infrastructure, Xtext-specific builder functionality and |
| several adjustments for N4JS and N4MF.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>The <literal>IBuilderState</literal> implementation is identical to the persisted Xtext index. No matter how many Xtext languages are supported |
| by the application, only a single <literal>IBuilderState</literal> instance is available in the application. Since we have one single <literal>IBuilderState</literal>, |
| we have one single persisted Xtext index throughout the application.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>For simplicity, the below description assumes we have only N4JS projects in the workspace and no other Xtext languages are |
| installed.</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>Major components:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><literal>XtextBuilder</literal> (inherits from Eclipse’s <literal>IncrementalProjectBuilder</literal>):</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>the actual incremental builder</simpara> |
| </listitem> |
| <listitem> |
| <simpara>note: Eclipse will create one instance of <literal>XtextBuilder</literal> per project at startup.</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara><literal>IBuilderState</literal> (Xtext specific; no Eclipse pendant):<?asciidoc-br?> |
| identical to the <literal>Xtext index</literal>, i.e. the globally shared, persisted instance of<?asciidoc-br?> |
| <literal>IResourceDescriptions</literal>.</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>Main workflow:</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>for each project that contains at least one resource that requires rebuilding, Eclipse will call the |
| project’s <literal>XtextBuilder</literal>.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>each <literal>XtextBuilder</literal> will perform some preparations and will then delegate to <literal>IBuilderState</literal> which will iterate over |
| all resources in the builder’s project that require rebuilding.</simpara> |
| </listitem> |
| </orderedlist> |
| <section xml:id="sec:Incremental_Builder_Overview__XtextBuilder"> |
| <title>XtextBuilder</title> |
| <simpara>Whenever a change in the workspace happens …​</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara><anchor xml:id="itm:start" xreflabel="[itm:start]"/> Eclipse will collect all projects that contain changed resources and compute a project-level build |
| order (using the <literal>build order</literal> of the workspace, see <literal>Workspace#getBuildOrder()</literal>, which is based on project dependencies)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>for the first <footnote><simpara>First, according to the build order.</simpara></footnote> project with changed resources, Eclipse will invoke |
| method <literal>IncrementalProjectBuilder#build(int,Map,IProgressMonitor)</literal> of the project’s <literal>XtextBuilder</literal><?asciidoc-br?> |
| (NOTE: from this point on, we are in the context of a <literal>current project</literal>)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>in <literal>XtextBuilder#build(int,Map,IProgressMonitor)</literal>:<?asciidoc-br?> |
| the builder creates an empty instance of <literal>ToBeBuilt</literal> (Xtext specific)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>in <literal>XtextBuilder#incrementalBuild(IResourceDelta,IProgressMonitor)</literal>:</simpara> |
| <orderedlist numeration="loweralpha"> |
| <listitem> |
| <simpara>The builder will iterate over all files in the project and for each will notify a <literal>ToBeBuiltComputer</literal> about the |
| change (added, updated, or deleted) which can then decide how to update the <literal>ToBeBuilt</literal> instance,</simpara> |
| </listitem> |
| <listitem> |
| <simpara>then forwards to <literal>#doBuild()</literal> .</simpara> |
| <simpara>Note: if user changes 1..* files in a single project but later more files in other, dependant projects |
| need to be built, the above step will happen for all projects, but will have an effect only for the first project that contains the actual file changes (i.e. in the standard case of saving a single file <literal>ToBeBuilt</literal> will always be non-empty for the <literal>first</literal> project, and always empty for the other, dependant projects; if a <literal>Save All</literal> is done, <literal>ToBeBuilt</literal> could be non-empty for later projects as well).</simpara> |
| </listitem> |
| </orderedlist> |
| </listitem> |
| <listitem> |
| <simpara>in <literal>XtextBuilder#doBuild(ToBeBuilt,IProgressMonitor,BuildType)</literal>:</simpara> |
| <orderedlist numeration="loweralpha"> |
| <listitem> |
| <simpara>first check if <literal>ToBeBuilt</literal> is empty AND global build queue does not contain URIs for current project → then abort (nothing to do here)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>creates instance of BuildData with:</simpara> |
| <orderedlist numeration="lowerroman"> |
| <listitem> |
| <simpara>name of current project (as string)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>newly created, fresh <literal>ResourceSet</literal></simpara> |
| </listitem> |
| <listitem> |
| <simpara>the <literal>ToBeBuilt</literal> (containing URIs of actually changed resources within current project, possibly filtered by <literal>ToBeBuiltComputer</literal>)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>the <literal>QueuedBuildData</literal> (an injected singleton)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>mode flag <literal>indexingOnly</literal> (only true during crash recovery)</simpara> |
| </listitem> |
| </orderedlist> |
| </listitem> |
| <listitem> |
| <simpara>invoke <literal>IBuilderState</literal> passing the <literal>BuildData</literal><?asciidoc-br?> |
| → updates itself (it is the global Xtext index) to reflect all changes in <literal>current project</literal>; validates and updates markers; runs transpiler (see below for details)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>invoke all registered <literal>IXtextBuilderParticipants</literal> (Xtext specific) for the <literal>current project</literal></simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>this is where normally we would do validation and run the transpiler; however, for performance reasons (do not load resource again) we already do this in the <literal>IBuilderState</literal> (this is the idea of the <literal>GenerateImmediatelyBuilderState</literal>)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>in our implementation, almost nothing is done here, except trivial stuff such as deleting files during clean build</simpara> |
| <simpara>At this point: returning from all methods.</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| </orderedlist> |
| </listitem> |
| <listitem> |
| <simpara>back in <literal>XtextBuilder#build(int,Map,IProgressMonitor)</literal>:<?asciidoc-br?> |
| → return with an array of IProjects; in our case: we return all other N4JSProjects referenced in the package.json of the project</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>important: these are <emphasis role="strong">not</emphasis> the projects that will be processed next: we need to continue with projects that depend on the current project, not with projects the current project depends on!</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Eclipse calls the returned projects <literal>interestingProjects</literal> and uses that as a hint for further processing; details not discussed here.</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara>continue with step <link linkend="itm:start">one</link>:<?asciidoc-br?> |
| Eclipse will invoke <literal>XtextBuilder#build(int,Map,IProgressMonitor)</literal> again for all other projects that have a dependency to the <literal>current project</literal> of the previous iteration, plus all remaining projects with changed resources.</simpara> |
| </listitem> |
| </orderedlist> |
| </section> |
| <section xml:id="sec:Incremental_Builder_Overview__IBuilderState"> |
| <title>IBuilderState</title> |
| <simpara>Invoked: once for each project containing a changed resource and dependant projects.<?asciidoc-br?> |
| Input: one instance of <literal>BuildData</literal>, as created by <literal>XtextBuilder</literal>, containing:</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>name of current project (as string)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>newly created, fresh <literal>ResourceSet</literal></simpara> |
| </listitem> |
| <listitem> |
| <simpara>the <literal>ToBeBuilt</literal></simpara> |
| <orderedlist numeration="loweralpha"> |
| <listitem> |
| <simpara>set of to-be-deleted URIs</simpara> |
| </listitem> |
| <listitem> |
| <simpara>set of to-be-updated URIs</simpara> |
| </listitem> |
| </orderedlist> |
| </listitem> |
| <listitem> |
| <simpara>the <literal>QueuedBuildData</literal>, an injected singleton maintaining the following values <footnote><simpara>These are not really input values but rather values changed during the following invocation of the IBuilderState that need to be carried over from one invocation to the next.</simpara></footnote>:</simpara> |
| <orderedlist numeration="loweralpha"> |
| <listitem> |
| <simpara>a queue of URIs per project (below called the <literal>global queue</literal>)<?asciidoc-br?> |
| (actually stored in <literal>QueuedBuildData#projectNameToChangedResource</literal>)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>a collection of <literal>all remaining URIs</literal><?asciidoc-br?> |
| (derived value: queued URIs of all projects + queues URIs not associated to a project (does not happen in N4JS))</simpara> |
| </listitem> |
| <listitem> |
| <simpara>a collection of <literal>pending deltas</literal> (always empty in N4JS; probably only used for interaction with Java resources)</simpara> |
| </listitem> |
| </orderedlist> |
| </listitem> |
| <listitem> |
| <simpara>mode flag <literal>indexingOnly</literal> (only true during crash recovery)</simpara> |
| </listitem> |
| </orderedlist> |
| <section xml:id="copy-and-update-xtext-index"> |
| <title>Copy and Update Xtext Index</title> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>in <literal>IBuilderState#update(BuildData,IProgressMonitor)</literal>:<?asciidoc-br?> |
| creates a copy of its <literal>ResourceDescriptionsData</literal> called <literal>newData</literal> <footnote><simpara>Once the build phase has ended, this copied and modified Xtext index will replace the actual state of the builder state and will be persisted on graceful application shutdown.</simpara></footnote></simpara> |
| </listitem> |
| <listitem> |
| <simpara>in <literal>AbstractBuilderState#doUpdate(…​)</literal>:<?asciidoc-br?> |
| updates <literal>newData</literal> by writing new resource descriptions into it.</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>Creates a new load operation (<literal>LoadOperation</literal>) instance from the <literal>BuildData#getToBeUpdated()</literal> and loads all entries. While iterating and loading the resource descriptions, it updates <literal>newData</literal> by registering new resource descriptions that are being created on the fly from the most recent version of the corresponding resources.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Adds these resources to the current project’s build queue. (<literal>BuildData#queueURI(URI uri)</literal>)</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara>for all to-be-deleted URIs given in <literal>ToBeBuilt</literal> in the <literal>BuildData</literal>, removes the corresponding <literal>IResourceDescription</literal> from <literal>newData</literal></simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><literal>ToBeBuilt#getAndRemoveToBeDeleted()</literal> returns all URIs that have been marked for deletion but not marked for update and will clear the set of to-be-deleted URIs in <literal>ToBeBuilt</literal>.</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| </orderedlist> |
| </section> |
| <section xml:id="build-state-setup-phase"> |
| <title>Build State Setup Phase</title> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>Calculates a set <literal>allRemainingURIs</literal> <footnote><simpara>This set of URIs will contain the URIs of all resources that are available in the copied Xtext index but not yet directly processed by the builder in the current build phase. These URIs will later be used as candidates for all resources that might be marked as affected ones and queued by the builder for forthcoming build phases.</simpara></footnote> as follows:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>Initially contains all resource URIs from <literal>newData</literal>.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>All URIs will be removed from it that are marked for update (<literal>BuildData#getToBeUpdated()</literal>).</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Finally, all URIs will be removed from it that are already queued for build/rebuild. (<literal>BuildData#getAllRemainingURIs()</literal>).</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara>Creates an empty set <literal>allDeltas</literal> of resource description deltas<?asciidoc-br?> |
| (c.f. <literal>IResourceDescription.Delta</literal>). <footnote><simpara>This set eventually represents all changes that were made during the current build phase. Note that <literal>allChanges</literal> might contain resource description deltas that do not represent an actual change, it is processed by the builder but the underlying information stored in the user data is still unchanged.</simpara></footnote></simpara> |
| </listitem> |
| <listitem> |
| <simpara><anchor xml:id="itm:processDeleted" xreflabel="[itm:processDeleted]"/> <emphasis role="strong">Process Deleted:</emphasis> for all to-be-deleted URIs, creates a delta where the old state is the current state of the resource and the new state is <literal>null</literal> and adds it to <literal>allDeltas</literal>.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Adds all <literal>pending deltas</literal> from <literal>QueuedBuildData</literal> to <literal>allDeltas</literal> (does not apply to N4JS).</simpara> |
| </listitem> |
| <listitem> |
| <simpara><emphasis role="strong"><anchor xml:id="itm:enqueueAffectedResources" xreflabel="[itm:enqueueAffectedResources]"/>Enqueue affected resources, part 1:</emphasis> adds to the <literal>global queue</literal> the URIs of all resources affected by the changes in <literal>allDeltas</literal>.</simpara> |
| <note> |
| <simpara>For N4JS, <literal>allDeltas</literal> always seems to be empty at this point, so this does nothing at all.</simpara> |
| </note> |
| </listitem> |
| <listitem> |
| <simpara>Creates an empty set <literal>changedDeltas</literal> for storing deltas that were modified by the build phase and represent an actual change. Unlike <literal>allDeltas</literal>, this set contains only those URIs that were processed by the builder - the underlying user data information contains the differences between the old and the new state.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Creates a new <literal>current queue</literal> and adds all URIs from the <literal>global queue</literal> that belong to the <literal>current project</literal>.</simpara> |
| </listitem> |
| </orderedlist> |
| </section> |
| <section xml:id="process-queued-uris"> |
| <title>Process Queued URIs</title> |
| <simpara>Processes all elements from the queue until it contains no more elements.</simpara> |
| <orderedlist xml:id="itm:loadRes" numeration="arabic"> |
| <listitem> |
| <simpara>Load the resource for the first/next URI on the current queue</simpara> |
| <note> |
| <simpara>In case of a move, the loaded resource could have a different URI!</simpara> |
| </note> |
| </listitem> |
| <listitem> |
| <simpara>Once the resource has been loaded, it removes its URI from the current queue to ensure it will not be processed again.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>If the loaded resource is already marked for deletion, stop processing this resource and continue with next URI from the current queue (go to step <link linkend="itm:loadRes">Load Res</link>) |
| <footnote><simpara>Note that deltas for to-be-deleted resources were already added to <literal>allDeltas</literal> upfront in step <link linkend="itm:processDeleted">Process Deleted</link>.</simpara></footnote></simpara> |
| </listitem> |
| <listitem> |
| <simpara>Resolves all lazy cross references in the loaded resource. This will trigger post-processing, including all type inference (c.f. <literal>ASTProcessor#processAST(…​)</literal>).</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Creates a delta for the loaded resource, including</simpara> |
| <orderedlist numeration="loweralpha"> |
| <listitem> |
| <simpara>a resource description based on the new state of the resource, wrapped into the <literal>EObject</literal>-based resource description (as with the Xtext index persistence in <literal>EMFBasedPersister#saveToResource()</literal>).</simpara> |
| </listitem> |
| <listitem> |
| <simpara>a resource description for the same resource with the state before the build process.</simpara> |
| </listitem> |
| </orderedlist> |
| </listitem> |
| <listitem> |
| <simpara>Adds this new delta to <literal>allDeltas</literal> and, if the delta represents a change (according to <literal>DefaultResourceDescriptionDelta#internalHasChanges()</literal>), also adds it to <literal>changedDeltas</literal>.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Adds the resource description representing the new state, stored in the delta, to <literal>newData</literal>, i.e. the copied <literal>ResourceDescriptionsData</literal>, replacing the old resource description of the loaded resource <footnote><simpara>This happens through a call to <literal role="language-n4js">CurrentDescriptions#register(Delta)</literal></simpara></footnote>.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>If the current queue is non-empty, go to step <link linkend="itm:loadRes">Load Res</link> and continue with the next URI in the current queue.</simpara> |
| </listitem> |
| </orderedlist> |
| </section> |
| <section xml:id="queueing-affected-resources"> |
| <title>Queueing Affected Resources</title> |
| <simpara>When the current queue contains no more URIs (all have been processed) …​</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara><anchor xml:id="itm:updateGlobalQueue" xreflabel="[itm:updateGlobalQueue]"/> <emphasis role="strong">Enqueue affected resources, part 2:</emphasis> add to the global queue URIs for all resources affected by the changes in <literal>changedDeltas</literal> <footnote><simpara>Unlike in step <link linkend="itm:enqueueAffectedResources">Enqueue Affected Resources</link>, we now use <literal>changedDeltas</literal> instead of <literal>allDeltas</literal> as a basis.</simpara></footnote>.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Returns from <literal>#doUpdate()</literal>, returning <literal>allDeltas</literal> (only used for event notification).</simpara> |
| </listitem> |
| <listitem> |
| <simpara>back in <literal>IBuilderState#update(BuildData,IProgressMonitor)</literal>:<?asciidoc-br?> |
| makes the <literal>newData</literal> the publicly visible, persistent state of the IBuilderState (i.e. the <literal>official</literal> Xtext index all other code will see).</simpara> |
| </listitem> |
| </orderedlist> |
| <simpara>We now provide some more details on how the global queue is being updated, i.e. steps <link linkend="itm:enqueueAffectedResources">Enqueue Affected Resources</link> and <link linkend="itm:updateGlobalQueue">Update Global Queue</link>. |
| Due to the language specific customizations for N4JS, this second resource-enqueuing phase is the trickiest part of the incremental building process and has the largest impact on how other resources will be processed and enqueued at forthcoming builder state phases.</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>If <literal>allDeltas</literal> is empty, nothing to do.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>If <literal>allDeltas</literal> contains at least one element, we have to check other affected resources by going through the set of all resource URIs (<literal>allRemainingURIs</literal>) calculated in in the beginning of the build process.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Assume we have at least one element in the <literal>allDeltas</literal> set, the latter case is true and we must check all elements whether they are affected or not. We simply iterate through the <literal>allRemainingURIs</literal> set and retrieve the old state of the resource description using the resource URI.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Once the resource description with the old state is retrieved, we check if it is affected through the corresponding resource description manager. Since we currently support two languages, we have two different ways for checking whether a resource has changed or not. One for package.json files and the other for the N4JS language related resources.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>The package.json method is the following: get all project IDs referenced from the <literal>candidate</literal> package.json and compare it with the container-project name of the package.json files from the <literal>deltas</literal>. The referenced IDs are the followings:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>tested project IDs,</simpara> |
| </listitem> |
| <listitem> |
| <simpara>implemented project IDs,</simpara> |
| </listitem> |
| <listitem> |
| <simpara>dependency project IDs,</simpara> |
| </listitem> |
| <listitem> |
| <simpara>provided runtime library IDs,</simpara> |
| </listitem> |
| <listitem> |
| <simpara>required runtime library IDs and</simpara> |
| </listitem> |
| <listitem> |
| <simpara>extended runtime environment ID.</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara>The N4JS method is the following:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>We consider only those changed deltas which represent an actual change (<literal>IResourceDescription.Delta#haveEObjectDescriptionsChanged()</literal>) and have a valid file extension (<literal>.n4js</literal>, <literal>.n4jsd</literal> or <literal>.js</literal>).</simpara> |
| </listitem> |
| <listitem> |
| <simpara>For each <literal>candidate</literal>, we calculate the imported FQNs. The imported FQNs includes indirectly imported names besides the directly imported ones. Indirectly imported FQNs are, for instance, the FQNs of all transitively extended super class names of a direct reference.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>We state that a <literal>candidate</literal> is affected if there is a dependency (for example name imported by a <literal>candidate</literal>) to any name exported by the description from a delta. That is, it computes if a candidate (with given <literal>importedNames</literal>) is affected by a change represented by the description from the delta.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>If a <literal>candidate</literal> is affected we have to do an additional dependency check due to the lack of distinct unique FQNs. If a project containing the delta equals with the project contained by the candidate, or if the project containing the candidate has a direct dependency to the project containing the delta, we mark a candidate as affected.</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara>If a candidate was marked as affected, it will be removed from the <literal>allRemainingURIs</literal> and will be added to the build queue.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>If a candidate has been removed from the <literal>allRemainingURIs</literal> and queued for the build, we assume its <literal>TModule</literal> information stored in the user data is obsolete. To invalidate the obsolete information, we wrap the delta in the custom resource description delta so whenever the <literal>TModule</literal> information is asked for, it will be missing. We then register this wrapped delta into the copied Xtext index, end the builder state for the actual project then invoke the Xtext builder with the next dependent project.</simpara> |
| </listitem> |
| </itemizedlist> |
| </section> |
| </section> |
| <section xml:id="sec:Incremental_Builder_Overview__Example"> |
| <title>Example</title> |
| <simpara>To conclude this section, we briefly describe the state of the above five phases through a simple example. Assume a |
| workspace with four N4JS projects: <emphasis>P1</emphasis>, <emphasis>P2</emphasis>, <emphasis>P3</emphasis> and <emphasis>PX</emphasis>. Each project has one single module with one single |
| publicly visible class. Also let’s assume project <emphasis>P2</emphasis> depends on <emphasis>P1</emphasis> and <emphasis>P3</emphasis> depends on <emphasis>P2</emphasis>. Project <emphasis>PX</emphasis> have |
| no dependencies to other projects. Project <emphasis>P1</emphasis> has a module <emphasis>A.n4js</emphasis> with a class <literal>A</literal>, project <emphasis>P2</emphasis> has one single |
| module <emphasis>B.n4js</emphasis>. This module has a public exported class <literal>B</literal> which extends class <literal>A</literal>. Furthermore, project <emphasis>P3</emphasis> has |
| one single module: <emphasis>C.n4js</emphasis>. This module contains one exported public class <literal>C</literal> which extends <literal>B</literal>. Finally, project |
| <emphasis>PX</emphasis> has a module <emphasis>X.n4js</emphasis> containing a class <literal>X</literal> that has no dependencies to any other classes. The figure below |
| picture depicts the dependencies between the projects, the modules and the classes as described above.</simpara> |
| <figure role="center"> |
| <title>Builder State Example</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/07_typeindex/images/builderStateExample.png"/> |
| </imageobject> |
| <textobject><phrase>builderStateExample</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara>For the sake of simplification, the table below describes a symbol table for all resources:</simpara> |
| <table frame="all" rowsep="1" colsep="1"> |
| <title>Resource Symbol Table</title> |
| <tgroup cols="2"> |
| <colspec colname="col_1" colwidth="50*"/> |
| <colspec colname="col_2" colwidth="50*"/> |
| <tbody> |
| <row> |
| <entry align="left" valign="top"><simpara>P1/src/A.n4js</simpara></entry> |
| <entry align="left" valign="top"><simpara>A</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>P2/src/B.n4js</simpara></entry> |
| <entry align="left" valign="top"><simpara>B</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>P3/src/C.n4js</simpara></entry> |
| <entry align="left" valign="top"><simpara>C</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>PX/src/X.n4js</simpara></entry> |
| <entry align="left" valign="top"><simpara>X</simpara></entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </table> |
| <simpara>Let assume auto-build is enabled and the workspace contains no errors and/or warnings. We make one simple modification |
| and expect one single validation error in class <literal>C</literal> after the incremental builder finished its processing; we delete the |
| method <literal>foo()</literal> from class <literal>A</literal>.</simpara> |
| <simpara>After deleting the method in the editor and saving the editor content, a workspace modification operation will run and |
| that will trigger an auto-build job. The auto-build job will try to build the container project <emphasis>P1</emphasis> of module <emphasis>A</emphasis>. Since |
| the project is configured with the Xtext builder command, a builder state update will be performed through the Xtext builder. |
| Initially, due to an Eclipse resource change event (we literally modify the resource from the editor and save it), the |
| <literal>ToBeBuilt</literal> instance wrapped into the <literal>BuildData</literal> will contain the URI of the module <emphasis>A</emphasis> marked for an update. When updating |
| the copied index content, module <emphasis>A</emphasis> will be queued for a build. While processing the queued elements for project <emphasis>P1</emphasis>, |
| module <emphasis>A</emphasis> will be processed and will be added to the <literal>allDeltas</literal> set. Besides that, it will be added to the <literal>changedDeltas</literal> |
| set as well. That is correct, because its <literal>TModule</literal> information has been changed after deleting the public <literal>foo()</literal> method. |
| When queuing affected resources, iterating through the set of <literal>allRemainingURIs</literal>, we recognize that module <emphasis>B</emphasis> is affected. |
| That is indeed true; module <emphasis>B</emphasis> imports the qualified name of class <literal>A</literal> from module <emphasis>A</emphasis> and project <emphasis>P2</emphasis> has a direct |
| dependency to <emphasis>P1</emphasis>. In this builder state phase, when building project <emphasis>P1</emphasis>, module <emphasis>C</emphasis> is not considered as affected. |
| Although class <literal>C</literal> from module <emphasis>C</emphasis> also imports the qualified name of class <literal>A</literal> from module <emphasis>A</emphasis>, project <emphasis>P3</emphasis> does not |
| have a direct dependency to project <emphasis>P1</emphasis>. When module <emphasis>B</emphasis> becomes enqueued for a forthcoming build phase, we assume its |
| <literal>TModule</literal> information is obsolete. We invalidate this <literal>TModule</literal> related user data information on the resource description |
| by wrapping the resource description into a custom implementation (<literal>ResourceDescriptionWithoutModuleUserData</literal>). Due to this |
| wrapping the resource description for module <emphasis>B</emphasis> will be marked as changed (<literal>IResourceDescription.Delta#haveEObjectDescriptionsChanged()</literal>) |
| whenever the old and current states are being compared.</simpara> |
| <simpara>The Eclipse builder will recognize (via <literal>IProjectDescription#getDynamicReferences()</literal>) that project <emphasis>P2</emphasis> depends on project <emphasis>P1</emphasis> |
| so the Xtext builder will run for project <emphasis>P2</emphasis> as well. At the previous phase we have enqueued module <emphasis>B</emphasis> for the build. |
| We will therefore run into a builder state update again. We do not have any resource changes this time, so <literal>ToBeBuilt</literal> will |
| be empty. Since <literal>ToBeBuilt</literal> is empty, we do not have to update the copied Xtext index state before the builder state setup |
| phase. As the result of the previous builder state, phase module <emphasis>B</emphasis> is already enqueued for a build. When processing <emphasis>B</emphasis> |
| we register it into the <literal>allDeltas</literal> set. That happens for each resource being processed by the builder state. But it will be |
| registered into the <literal>changedDeltas</literal> because we have previously wrapped module <emphasis>B</emphasis> into a customized resource description delta |
| to hide its obsolete <literal>TModule</literal> related user data information. Based on the builder state rules and logic described above, |
| module <emphasis>C</emphasis> will be marked as an affected resource, will be queued for build and will be wrapped into a customized resource |
| description delta to hide its <literal>TModule</literal> related user data information.</simpara> |
| <simpara>In the next builder state phase, when building project <emphasis>P3</emphasis>, we apply the same logic as we applied for project <emphasis>P2</emphasis>. The |
| builder state will process module <emphasis>C</emphasis> and will update the Xtext index state. No additional resources will be found as |
| affected ones, nothing will be queued for build. The build will terminate, since there were no changed <literal>IResource</literal> instances |
| and the build queue is empty.</simpara> |
| <simpara>The outcome of the incremental build will be a workspace that contains exactly one validation error. The error will be |
| associated with module <emphasis>C</emphasis> which was exactly our expectation, however, we have to clarify that transitive <emphasis>C</emphasis> dependency |
| was built due to wrong reasons. Module <emphasis>C</emphasis> was build because we wrapped module <emphasis>B</emphasis> to hide its user data information and |
| not because it imports and uses class <literal>A</literal> from module <emphasis>A</emphasis> which should be the logical and correct reason.</simpara> |
| </section> |
| </section> |
| <section xml:id="dirty-state-handling" role="language-n4js"> |
| <title>Dirty state handling</title> |
| <simpara>When two or more (N4)JS files are opened in editors and one of them is changed but without persisting this change the other |
| open editors should be notified and if this change breaks (or heals) references in one of the other open resources their editors |
| should updated so that warn and error markers are removed or added accordingly.</simpara> |
| <simpara>When there are changes in the currently open editor these changes are propagated to all other open editors. Each Xtext editor has |
| got its own resource set. The <literal>N4JSUpdateEditorStateJob</literal> runs for each open editor different from the editor where the changes have |
| been made. In those editors the affected resources are unloaded and removed from the resource set. Then the Xtext resource of |
| these editors is reparsed. After reparsing scoping and linking is invoked again, but now the references resources are rebuild |
| as <literal>EObjectDescription</literal>s. The <literal>N4JSResource</literal> holds its own content that only contains 1..n slots when proxified. |
| <literal>N4JSTypeScope.getSingleElement</literal> (called when resolving cross references and the linked element should be returned) will return the |
| <literal>EObjectDescription</literal> created from the <literal>ModuleAwareContentsList</literal> in <literal>N4JSResource</literal>, that contains the first slot as proxy and the other |
| slots as types. <xref linkend="fig:dirty_state_handling1"/> shows the flow to trigger the <literal>N4JSUpdateEditorStateJob</literal> and <xref linkend="fig:dirty_state_handling2"/> |
| shows the sequence logic of the <literal>N4JSUpdateEditorStateJob</literal> in detail.</simpara> |
| <figure xml:id="fig:dirty_state_handling1" role="center"> |
| <title>Sequence Diagram: Dirty State, Trigger <literal>N4JSUpdateEditorStateJob</literal></title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/07_typeindex/images/dirty_state_handling1.png"/> |
| </imageobject> |
| <textobject><phrase>dirty state handling1</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <figure xml:id="fig:dirty_state_handling2" role="center"> |
| <title>Sequence Diagram: Dirty State, <literal>N4JSUpdateEditorStateJob</literal> in Detail</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/07_typeindex/images/dirty_state_handling2.png"/> |
| </imageobject> |
| <textobject><phrase>dirty state handling2</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara>A concrete example should illustrate the behaviour of the dirty state handling in conjunction with fully and partial loading |
| of resources:</simpara> |
| <simpara>Let A.js as above, and B.js as follows:</simpara> |
| <programlisting language="n4js" linenumbering="unnumbered">import A from "A.js" |
| export class B {}</programlisting> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>assume is opened and loaded: is created with</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>is filled with a special proxy to resolve the <link linkend="AC">AST</link> of A only if needed.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>will be set to type A, loaded from <literal>EObjectDescription</literal> of A.js/A</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara><link linkend="AC">AST</link> of A.js is to be accessed, e.g., for displaying JSDoc. A.js is not opened in an editor! is modified as follows:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>is filled with <link linkend="AC">AST</link>, i.e. proxy in 0 is resolved</simpara> |
| </listitem> |
| <listitem> |
| <simpara>is updated with parsed type: 1) proxify , 2) unload (remove from content), 3) reload with parsed types again</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara>Assume now that A.js is opened and edited by the user.</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>Reconceiler replaces with modified <link linkend="AC">AST</link></simpara> |
| </listitem> |
| <listitem> |
| <simpara>LazyLinker updates </simpara> |
| </listitem> |
| <listitem> |
| <simpara>is proxified</simpara> |
| </listitem> |
| <listitem> |
| <simpara>B <literal>searches</literal> for A and finds updated </simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| </orderedlist> |
| <simpara><emphasis>Each opened Xtext editor has got its own resource set!</emphasis> Such a resource set contains the resource for the currently edited |
| file in the first place. When starting editing the file, the resource is reparsed , reloaded and linking (resolving the cross |
| references) is done. By resolving the cross references <literal>N4JSTypeScope</literal> is used and now the URIs of the linked elements are belonging |
| to resources not contained in the resource set of the editor so these resources a create in the resource set and their contents |
| is loaded from the resource descriptions via |
| <literal>N4JSResource.loadFromDescription</literal> .</simpara> |
| <simpara>When the resource content is loaded from the existing resource description available from Xtext index the first slot is set to be |
| a proxy with name <literal>#:astProxy</literal>. |
| After that for all exported EObject descriptions of that resource description the user data is fetched and deserialized to types |
| and these types are added to the slots of the resource in order they were exported. But the order is not that important anymore.</simpara> |
| <simpara>As the resource set for the editor is configured to use the DirtyStateManager ( <literal>ExternalContentSupport.configureResourceSet(resourceSet, dirtyStateManager)</literal> ), |
| all other open editors will be notified by changes in the current editor. This is done by <literal>N4JSDirtyStateEditorSupport</literal> that schedules |
| a <literal>N4JSUpdateEditorStateJob</literal> that create a new resource description change event.</simpara> |
| <simpara>Via <literal>isAffected</literal> and <literal>ResourceDescription.getImportedNames</literal> it is determine if a change in another resource affects this resource.</simpara> |
| <simpara>Before loading the resource always <literal>N4JSDerivedStateComputer.installDerivedState</literal> is execute that, as we learned earlier, is responsible |
| for creating the types in the resource.</simpara> |
| <simpara>On every change to a N4JS file that requires a reparse the <literal>N4JSDerivedStateComputer.discardDerivedState</literal> is executed. This method do an |
| unload on every root element at the positions 1 to n. In the <literal>N4JSUnloader</literal> all contents of the this root elements are proxified (i.e. |
| there is a proxy URI set to each of them) and the references to the <link linkend="AC">AST</link> are set to null (to avoid notifications causing |
| concurrent model changes). The proxification indicates for all callers of these elements, that they have to reload them. Afterwards |
| it discards the complete content of the resource. The content is build up again with the reparse of the N4JS file content.</simpara> |
| <simpara>As each editor has its own resource set, only the resource belonging to the current editor is fully loaded. All other referenced |
| resources are only partially loaded, i.e. only the slot 1 of these resources are loaded (i.e. the types model elements) in this |
| resource set. Linking is done only against these types model elements. Synchronization between the resource sets of multiple open |
| editors is done via update job as described above.</simpara> |
| <section xml:id="use-case-restoring-types-from-user-data"> |
| <title>Use case: Restoring types from user data</title> |
| <itemizedlist> |
| <listitem> |
| <simpara>Use case: referencing resources in editor: This has been described already in context of dirty state handling</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Use case: referencing resources from JAR: This is still to be implemented.% taskIDE-37</simpara> |
| </listitem> |
| </itemizedlist> |
| </section> |
| <section xml:id="use-case-updating-the-xtext-index"> |
| <title>Use case: Updating the Xtext index</title> |
| <simpara>When a N4JS file is changed in way that requires reparsing the file, the underlying resource is completely unloaded and loaded again. |
| By this the also the elements at the Xtext index are recreated again, belonging to this resource (i.e. new entries for new elements |
| in the resource, update index elements of changed elements, delete index entries for deleted elements).</simpara> |
| <simpara>When Eclipse is closed the Xtext index is serialized in a file.</simpara> |
| <informalfigure role="center"> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/07_typeindex/images/index_serialization.png"/> |
| </imageobject> |
| <textobject><phrase>index serialization</phrase></textobject> |
| </mediaobject> |
| </informalfigure> |
| <simpara>When starting Eclipse again, the Xtext index is restored from this file:</simpara> |
| <informalfigure role="center"> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/07_typeindex/images/loading_existing_index.png"/> |
| </imageobject> |
| <textobject><phrase>loading existing index</phrase></textobject> |
| </mediaobject> |
| </informalfigure> |
| </section> |
| </section> |
| </chapter> |
| <chapter xml:id="_project-model"> |
| <title>Project Model</title> |
| <section xml:id="sec:Package_json"> |
| <title>Package.json File</title> |
| <simpara>See [<link linkend="N4JSSpec">N4JSSpec</link>] for the format specification of N4JS-specific package.json files. |
| Based on the JSON-model-based AST that can be parsed from the package.json file, we transform the information that can be extracted into an instance of the N4JS-specific ProjectDescription model. |
| This model is defined by means of EMF, the Xcore file is found in the N4JS model bundle.</simpara> |
| <informalfigure xml:id="fig:projectDescriptionModel"> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/08_projectModel/images/cd_projectDescription.svg"/> |
| </imageobject> |
| <textobject><phrase>Project Description Model</phrase></textobject> |
| </mediaobject> |
| </informalfigure> |
| </section> |
| <section xml:id="_accessing-project-information"> |
| <title>Accessing Project Information</title> |
| <simpara>The information in the package.json files is parsed into memory at runtime, e.g. within Eclipse or in headless mode. It is made available via the <literal>IN4JSCore</literal> facade that provides a high-level abstraction when working with the project structure. This facade has two implementations:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>One implementation is backed by the file system, e.g. <literal>java.io.File</literal> and used in a headless environment</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Another implementation uses the Eclipse resource model (<literal>IProject</literal>, <literal>IFolder</literal>, <literal>IFile</literal>) to describe the contents of an <literal>IN4JSProject</literal>. This implementation is automatically kept in sync whenever the contents of a package.json file is edited by a user. It is also maintained as dynamic project information to make the Eclipse workspace aware of the declared dependencies. That is, Eclipse projects know about the project references in the package.json file.</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>The project model is mimicing the handle based services of Eclipse’s <literal>JavaCore</literal>, the handles are often represented as EMF <literal>URIs</literal> though. Therefore the API allows to retrieve a source folder for a given N4JS resource, all the libraries that are configured for a project or simply the project’s name or the information if it exists. Subsequent components in the processing chain like the scope provider can leverage the API to deduce the container configuration and visiblity constraints. Project references are resolved transparently.</simpara> |
| <simpara>The Xtext index participation is implemented by means of the <literal>N4JSToBeBuiltComputer</literal> and the <literal>N4JSAllContainersState</literal>. These classes provide access to the container configuration.</simpara> |
| <simpara>The precedence for the dependency resolution follows this lookup chain:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>An accessible <literal>IN4JSProject</literal> with the given name is located in the workspace.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>The library manager provides access to the requested project.</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>Accordingly, the type lookup follows the same chain due to the container configuration that is deduced from the package.json configuration. Of course is prefers locally defined type that can be found in the current project over types that are located in referenced projects.</simpara> |
| <section xml:id="sec:IN4JSCore"> |
| <title>IN4JSCore</title> |
| <simpara>Facade analogous to org.eclipse.jdt.core.JavaCore. |
| It is used look up the project or source container where a resource URI belongs to. |
| It also provides some helper methods to retrieve information from the package.json file. |
| N4JSRuntimeCore uses the file system and thus uses java.io to access files and folders. |
| N4JSEclipseCore uses the Eclipse workspace and thus uses org.eclipse.resources API. |
| Both Core implementations act as wrapper for N4JSModel resp. EclipseN4JSModel. |
| Instances of the <literal>IN4JSCore</literal> are obtained as usually via dependency injection, e.g. <literal>@Inject IN4JSCore n4jsCore</literal>.</simpara> |
| </section> |
| <section xml:id="sec:N4JSModel"> |
| <title>N4JSModel</title> |
| <simpara>N4JSModel uses FileBasedWorkspace to load and access the project description from the package.json file and create wrappers for projects (N4JSProject) and source containers (N4JSSourceContainer).</simpara> |
| <simpara>A source container is a wrapper for a file system that has been marked as source folder in the package.json file. |
| For determination of the current project a given EMF URI pointing to the project path is used. |
| In N4JSModel this location is directly wrapped in N4JSProject. |
| In N4JSModel a given EMF URI is resolved to a source container by using the specified relative source paths from the package.json file and file system based project location. |
| If the EMF URI converted to a file URI starts with the absolute source folder path a N4JSProjectSourceContainer is created and returned for that EMF URI.</simpara> |
| <simpara>In EclipseN4JSModel (that extends N4JSModel) the last segment of the URI is assumed to be the project name and via the EclipseBasedN4JSWorkspace that wraps the Eclipse workspace a project with that name is tried to be resolved. |
| This IProject is than wrapped in N4JSEclipseProject. |
| In EclipseN4JSModel a given EMF URI is resolved to an org.eclipse.core.resources.IResource belonging to the IWorkspaceRoot. |
| That resource is wrapped in EclipseSourceContainer.</simpara> |
| <simpara>N4JSModel is also used to retrieve project dependencies wrapped as N4JSProject.</simpara> |
| </section> |
| <section xml:id="sec:N4JSWorkspace"> |
| <title>N4JSWorkspace</title> |
| <simpara>The FileBasedWorkspace and EclipseBasedN4JSWorkspace should only be accessed by N4JSModel resp. EclipseN4JSModel as they know the contract for the URI scheme. |
| The FileBasedWorkspace creates AbstractTreeIterator for the direct contents of a source container, so that their children can be navigated by this as well. |
| It then filters out all directories and returns an iterator of all files as EMF URIs.</simpara> |
| <simpara>The EclipseBasedN4JSWorkspace wraps the IWorkspaceRoot.</simpara> |
| <simpara>Fetching the project description read out from the package.json file is cached in both workspace implementations. |
| In FileBasedWorkspace the LazyProjectDescriptionHandle is used as proxy and in EclipseBasedN4JSWorkspace the ProjectDescriptionLoadListener is used to invalidate the cache when the package.json file has been changed. |
| ProjectDescriptionLoadListener also ensures that dependent projects are considered by the Eclipse builder by setting dynamic dependencies in the project meta data. |
| It also updates these project dependencies if it is required and recalculates all source containers.</simpara> |
| <simpara>In the tests another implementation, MockWorkspace, is used, that provides a dummy project description.</simpara> |
| </section> |
| <section xml:id="sec:N4JSProject"> |
| <title>N4JSProject</title> |
| <simpara>A N4JSProject represents a configured project as defined in the package.json file. |
| So in principles it wraps access to N4JSModel (and so to containing source containers, libraries and so on) |
| and to some information from the package.json file directly (like defined module filters, vendorId and others). |
| It is also used to compare depending projects.</simpara> |
| <simpara>N4JSProject is the runtime representation while N4JSEclipseProject represents a project in the Eclipse workspace. |
| N4JSProject is identified by its EMF location URI while N4JSEclipseProject wraps the underlying org.eclipse.core.resources.IProject.</simpara> |
| <simpara>In tests MockProject is used.</simpara> |
| </section> |
| <section xml:id="sec:SourceContainer"> |
| <title>SourceContainer</title> |
| <simpara>A source container contains either source files for production, tests or external declarations. |
| By default all these files resp. their containing types will be exported to the Xtext index. |
| A source container belongs exactly to one project it is identified by its project relative location.</simpara> |
| <simpara>A N4JSProjectSourceContainer is a container that contains unpacked n4js, js and n4jsd files that can be modified. |
| By default all these files are syntactically and semantically validated (this can be configured by the use of module filters). |
| Except for n4jsd files, all files are compiled by the configured compilers.</simpara> |
| <simpara>EclipseSourceContainer specializes N4JSProjectSourceContainer to work on top of the Eclipse resources API. |
| Thus besides the location it also wraps the IFolder.</simpara> |
| <simpara>The IFile was chosen instead of using the java.io.File because changes to an IFile (and IResource in general) trigger a workspace change event so that the Xtext builder is triggered properly.</simpara> |
| </section> |
| <section xml:id="sec:N4JSProjectsStateHelper"> |
| <title>N4JSProjectsStateHelper</title> |
| <simpara>Calculates the visible containers for a given container, where containers are source containers within the project as well as source containers of other depending projects in workspace. |
| The calculated handles are cached and invalidated if the project description file has changed or the project has been closed or reopened.</simpara> |
| </section> |
| </section> |
| <section xml:id="sec:Caching"> |
| <title>Caching</title> |
| <simpara>Caching is heavily used in the ExternalLibraryWorkspace and the N4JSProjectsStateHelper. |
| The ExternalLibraryWorkspace relies on caching to provide data about all installed npms, their locations, names, shadowing, dependencies and so on. |
| The caching of the ExternalLibraryWorkspace is implemented in the ExternalProjectMappings which inspects all external locations and builds all necessary mappings. |
| The set of mappings start from a simple list of all npms, include mappings that map from location or name to a N4JSExternalProject, or give reduced set of projects.</simpara> |
| <section xml:id="_caching-of-externallibraryworkspace"> |
| <title>Caching of ExternalLibraryWorkspace</title> |
| <simpara>A reduced set of projects is used since not all npms are actually necessary projects for the N4JS IDE. |
| Most transitively installed plain-JS npms are of no interest since they are completely invisible to the user. |
| The reduced set of projects always consists of all user workspace projects and all shipped libraries. |
| From the set of all installed npms, only those are necessary that are dependencies of a non-plain-JS projects. |
| Shadowed projects are also not included in the reduced set of npms.</simpara> |
| <simpara>To access projects that are not included in the reduced set of npms, the ExternalProjectMappings provides some collections that contain complete set of npms. |
| Additionally, some mappings also provide information about not necessary npms. |
| Note that mappings that use the project name as a key naturally cannot provide information about shadowed projects.</simpara> |
| <simpara>The mapping cache is updated every time a refresh is triggered (e.g. at startup or by hitting F5). |
| Also, every action of the library manager (such as installing or registering npms) triggers a refresh.</simpara> |
| </section> |
| <section xml:id="_caching-of-n4jsprojectsstatehelper"> |
| <title>Caching of N4JSProjectsStateHelper</title> |
| <simpara>The N4JSProjectsStateHelper uses the MultiCleartriggerCache for caching information about projects of the user workspace. |
| The EclipseBasedN4JSWorkspace does not caching at all, but provides information about project descriptions which is expensive to compute on the fly. |
| Hence this information is cached in the MultiCleartriggerCache and updated every time a project description changes, is added or removed.</simpara> |
| <simpara>Sometimes, a project description is rendered invalid as a side effect of a change on another project description. |
| In this case, the cache of both of project descriptions has to be updated. |
| The implementation to cope with these side effects uses the MultiCleartriggerCache which allows to set multiple triggers that will clear a cached object.</simpara> |
| <simpara>However, it seems reasonable to align the caching of the user workspace to the caching of the external workspace. |
| The reason is that caching of user workspace information such as N4JSProjects would increase build performance significantly. |
| This is since as of now, projects (and information about all their source containers) is computed on the fly, that causes thousands of expensive calls to the file system.</simpara> |
| </section> |
| </section> |
| <section xml:id="sec:WildcardPathFilter"> |
| <title>WildcardPathFilter</title> |
| <simpara>This class encapsulates the logic to resolve (wildcard containing) paths against the file system. |
| With the method matchPath it is also possible to resolve a path without using the file system. |
| This class is also able to resolve relative navigation in paths.</simpara> |
| </section> |
| <section xml:id="sec:ProjectUtils"> |
| <title>ProjectUtils</title> |
| <simpara>ProjectUtils provides additional methods for providing information only required in compilation, e.g. like file and module descriptor. |
| It uses IN4JSCore to retrieve the information of output path and whether module wrapping is required for a given file.</simpara> |
| </section> |
| </chapter> |
| <chapter xml:id="_binding"> |
| <title>Binding</title> |
| <warning> |
| <simpara>This section may be outdated!</simpara> |
| </warning> |
| <section xml:id="sec:Binding_Design_Rationale" role="language-n4js"> |
| <title>Design Rationale</title> |
| <simpara>Binding references to declarations follows the Xtext mechanism based on local <literal>N4JSScopeProvider</literal> and a global <literal>N4JSGlobalScopeProvider</literal> scope providers. The basic question is: to which elements are references bound to. This in particular interesting for all kind of type declarations, including functions as they are interpreted as types. These declarations are Janus-faced: On the one side, they are targets of type references as <literal>Type</literal>, and on the other side they can also be target of identifier references bound to some so called <literal>IdentifiableElement</literal>. As explained in <xref linkend="_type_index"/>, special type objects (<literal>TClass</literal> etc.) are created from the original declarations. These type objects are used as targets for both kind of references. The following table summarizes the reference-target relations relevant for N4JS (not the standalone type grammar).</simpara> |
| <table frame="all" rowsep="1" colsep="1"> |
| <title>N4JS Cross References</title> |
| <tgroup cols="2"> |
| <colspec colname="col_1" colwidth="50*"/> |
| <colspec colname="col_2" colwidth="50*"/> |
| <thead> |
| <row> |
| <entry align="left" valign="top">Reference</entry> |
| <entry align="left" valign="top">Target Type</entry> |
| </row> |
| </thead> |
| <tbody> |
| <row> |
| <entry align="left" valign="top" namest="col_1" nameend="col_2"><simpara><emphasis role="strong">N4JS</emphasis></simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>ImportDeclaration.importedModule</simpara></entry> |
| <entry align="left" valign="top"><simpara>TModule</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>NamedImportSpecifier.importedElement</simpara></entry> |
| <entry align="left" valign="top"><simpara>types.IdentifiableElement</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>IdentifierRef.id</simpara></entry> |
| <entry align="left" valign="top"><simpara>types.IdentifiableElement</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>ParameterizedPropertyAccessExpression.property</simpara></entry> |
| <entry align="left" valign="top"><simpara>types.TMethod</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>PropertyAccessExpression.property</simpara></entry> |
| <entry align="left" valign="top"><simpara>types.TMember</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>N4Getter/N4SetterDeclaration.field</simpara></entry> |
| <entry align="left" valign="top"><simpara>N4FieldDeclaration</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>Continue/Break-Statement.label</simpara></entry> |
| <entry align="left" valign="top"><simpara>LabelledStatement</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top" namest="col_1" nameend="col_2"><simpara>Type Expressions</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>ParameterizedTypeRef.declaredType</simpara></entry> |
| <entry align="left" valign="top"><simpara>Type</simpara></entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </table> |
| <simpara><xref linkend="fig:cd_scoping"/> gives an overview over the most important classes in the scoping package, with the <literal>N4JSScopeProvider</literal> and the used customized scopes created by the scope providers.</simpara> |
| <figure xml:id="fig:cd_scoping" role="center"> |
| <title>Overview Scoping Package</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/09_binding/images/cd_scoping.svg"/> |
| </imageobject> |
| <textobject><phrase>cd scoping</phrase></textobject> |
| </mediaobject> |
| </figure> |
| </section> |
| <section xml:id="sec:Binding_to_Members" role="language-n4js"> |
| <title>Binding to Members</title> |
| <simpara>Members of different types, such as classes and also record types or enumerations, are bound using the <literal>MemberScopeProvider</literal>. This often returns a <literal>MemberScope</literal>, which directly works on the members. Most types with members are implemented by subclasses of <literal>ContainerType</literal>, using <literal>CollectMembersHelper</literal> to collect all members and <literal>FindMemberHelper</literal> for retrieving a member by its name via <literal>ContainerTypes</literal>. Ensure that when types with members are added to override appropriate methods in all of these related classes (e.g., <literal>CollectMembersHelper</literal>, <literal>AbstractHierachyTraverser</literal> and <literal>FindMemberHelper</literal> uses polymorphic dispatch to handle different subtypes – so you won’t be able to find a member if you do not adjust these helpers).</simpara> |
| </section> |
| <section xml:id="sec:Binding_Getter_Setter"> |
| <title>Getter / Setter Binding</title> |
| <simpara>For customized binding of getters / setters, see <link linkend="sec:Field_Accessors">Accessors</link>.</simpara> |
| </section> |
| <section xml:id="chap:Statics"> |
| <title>Static Member Binding</title> |
| <simpara>For customized binding of static members, see <link linkend="sec:Static_Members">Static Members</link>.</simpara> |
| </section> |
| <section xml:id="sec:Binding_Enumeration"> |
| <title>Enumeration Literals Binding</title> |
| <itemizedlist> |
| <listitem> |
| <simpara>introduced new type ref EnumTypeRef: it behaves comparable to ClassifierTypeRef, but with the difference that the MemberScopeProvider filters for a given EnumTypeRef filters all literals of the contained TEnum (in comparison the MemberScopeProvider filters for a given ClassifierTypeRef all static members of the contained classifier)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>it isn’t possible to access literals on a enumeration literal itself, although this literal is typed as TEnum (that contains TEnumLiterals)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>as there are currently no additional fields and operations for enumeration literals defined (in Java there is name and value()), the scope for literals is currently empty</simpara> |
| </listitem> |
| </itemizedlist> |
| </section> |
| <section xml:id="sec:Accessibility_of_types_and_members" role="language-n4js"> |
| <title>Accessibility of types and members</title> |
| <simpara>Member access and type access has to be constrained and validated against the accessibility rules of N4JS. Therefore, the scoping annotates certain elements as erroneous to detect invalid references.</simpara> |
| <simpara>Basically two different approaches are used to implement that behavior:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>The <literal>VisibilityAwareTypeScope</literal> and <literal>VisibilityAwareMemberScope</literal> decorate an existing scope to validate the result on access. This allows to lazily check the visibility of the returned element. If it is not accessible, it is wrapped in a <literal>AbstractDescriptionWithError</literal> which will be indentified as such by the <literal>ErrorAwareLinkingService</literal>. Before the binding is resolved and the EMF proxy is replaced, the error message is used to create an EMF diagnostic.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>For other cases, the scopes are produced differently, e.g. if all elements are easily enumerable and have to be collected before the scope is created (e.g. for imported elements), the scoped elements are validated eagerly to put them into the correct layer of scopes. That is, the valid descriptions may shadow the invalid description. Since there are more error conditions for these cases, e.g. duplicate imports and similar cases, the accessibility is checked before the concrete member is accessed. All the instances <literal>AbstractDescriptionWithError</literal> are put into the <literal>MapBasedScope</literal> immediately.</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>In that sense, accessibility checks are basically implemented as decorators for the scoping itself. Bindings are established but flagged as errors.</simpara> |
| <simpara>Default visibility of members is calculated in <literal>Types.xcore</literal> (in <literal>getTypeAccessModifier</literal> and <literal>getMemberAccessModifier</literal> etc.). Visibility is checked in <literal>org.eclipse.n4js.scoping.accessModifiers.MemberVisibilityChecker</literal> and validators.</simpara> |
| </section> |
| <section xml:id="sec:Member_Scope_Example" role="language-n4js"> |
| <title>Member Scope Example</title> |
| <simpara>In this section, we are going to have a look at the creation process of <literal>MemberScope</literal>.</simpara> |
| <formalpara> |
| <title>C.n4js</title> |
| <para> |
| <screen>export public class C { |
| private m1: int; |
| public m2: int; |
| }</screen> |
| </para> |
| </formalpara> |
| <formalpara> |
| <title>Test.n4js</title> |
| <para> |
| <screen>import { C } from "C"; |
| |
| let c: C = new C(); |
| c.m1; // Error -> The field m1 is not visible |
| c.m2; // OK -> m2 is visible at this context</screen> |
| </para> |
| </formalpara> |
| <simpara>Assume that we need to figure out to which element the <literal>ParameterizedPropertyAccessExpression c.m1</literal> in the <literal>ExpressionStatement c.m1</literal> binds to. To answer this question, <literal>N4JSScopeProvider.getScope(context, reference)</literal> is triggered whereby <literal>context</literal> is the <literal>ParameterizedPropertyAccessExpression</literal> and <literal>reference</literal> is <literal>EReference property</literal> (<literal>property</literal> is the cross-reference element defined in <literal>ParameterizedPropertyAccessExpression</literal> 's grammar).</simpara> |
| <simpara><literal>N4JSScopeProvider.getScope(context, reference)</literal> does not implement the scoping but delegates to corresponding methods based on the type of <literal>context</literal>. In our example, since <literal>context</literal> is a <literal>ParameterizedPropertyAccessExpression</literal>, the scoping logic is delegated to the method that creates a <emphasis role="strong">MemberScope</emphasis> for the context <literal>ParameterizedPropertyAccessExpression c.m1</literal> based on the receiver type of <literal>c</literal> which is class <literal>C</literal>. |
| The resulting scope instance returned by <literal>N4JSScopeProvider.getScope()</literal> in our example is of type <literal>TypingStrategyAwareMemberScope</literal> as shown in <xref linkend="fig:memberscope-example"/> .</simpara> |
| <figure xml:id="fig:memberscope-example"> |
| <title>Member scope hierarchy</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/09_binding/images/memberscope_example.svg" align="center"/> |
| </imageobject> |
| <textobject><phrase>memberscope example</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara>In the hierarchy, the top-level scope is the NULL scope. Directly below the NULL scope is a MemberScope which contains all members of <literal>N4Object</literal> since the class <literal>C</literal> implicitly inherits <literal>N4Object</literal>. The other <literal>MemberScope</literal> instance beneath contains all members of the class <literal>C</literal> <emphasis role="strong">regardless of their visibility</emphasis>. These members are <literal>m1</literal> and <literal>m2</literal>. While <literal>m2</literal> is can be accessed by <literal>c.m2</literal>, <literal>m1</literal> it not visible at <literal>c.m1</literal>. The <literal>VisibilityAwareMemberScope</literal> implements precisely this behavior. In particular, it returns all members of <literal>C</literal> that are visible at the current <literal>context</literal> (here the element <literal>m2</literal>), while wrapping non-visible members (here the element <literal>m</literal>) in <literal>InvisibleMemberDescription</literal> instances. These <literal>InvisibleMemberDescription</literal> instances of type <literal>IEObjectDescriptionWithError</literal> contain issue code and error message related to accessibility problems and are recognized during the error-aware linking phase done by <literal>ErrorAwareLinkingService</literal>. It is worth to emphasize the motivation behind use of <literal>IEObjectDescriptionWithError</literal> is to provide more informative error messages to the user other than <emphasis>Cannot reference element…​</emphasis> Another example of <literal>IEObjectDescriptionWithError</literal> is <literal>WrongWriteAccessDescription</literal> that is used when we, try to write to a getter and no corresponding setter exists.</simpara> |
| </section> |
| <section xml:id="sec:Scoping_for_Members_of_Composed_Type_Explained" role="language-n4js"> |
| <title>Scoping for Members of Composed Type (Union/Intersection) Example</title> |
| <simpara>In this section, we will have a look at how scoping is implemented for composed type, i.e. union or intersection type with an example of union type. Intersection is done similarly. Before reading this, it is strongly recommended to read <xref linkend="sec:Member_Scope_Example"/> first.</simpara> |
| <formalpara> |
| <title>Defs.n4js</title> |
| <para> |
| <screen>export public class C { |
| private m1: int; |
| public m2: int; |
| } |
| |
| export public class D { |
| private m1: int; |
| get m2(): int { return 42; }; |
| }</screen> |
| </para> |
| </formalpara> |
| <formalpara> |
| <title>Test.n4js</title> |
| <para> |
| <screen>import { C, D } from "Defs"; |
| |
| let cud : C|D; |
| |
| cud.m2 = 10;</screen> |
| </para> |
| </formalpara> |
| <simpara>Assume that we need to find out to what element the <literal>ParameterizedPropertyAccessExpression cud.m2</literal> in the <literal>ExpressionStatement cud.m2</literal> binds to. |
| This is a question for scoping. Since the receiver type of <literal>cud</literal> is a union type <literal>C|D</literal>, a <literal>UnionMemberScope</literal> is created that contains two subscopes, each of which corresponds to an individual type in the union. The resulting scope hierarchy is graphically depicted in <xref linkend="fig:unionmemberscope-example"/>.</simpara> |
| <figure xml:id="fig:unionmemberscope-example"> |
| <title>Union member scope hierarchy</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/09_binding/images/unionmemberscope_example.svg" align="center"/> |
| </imageobject> |
| <textobject><phrase>unionmemberscope example</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara>The two subscopes are of type <literal>TypingStrategyAwareMemberScope</literal> and created exactly the same way as described in <xref linkend="sec:Member_Scope_Example"/>. |
| The <literal>UnionMemberScope</literal> instance contains a list of subscopes for all types involved in the union and is responsible for constructing an <literal>IEObjectDescription</literal> instance for <literal>m2</literal> by merging all members of the name <literal>m2</literal> found in all subscopes. |
| Merging members requires considering a variety of combinations (fields, setters getters, optional/variadic parameters etc.) and thus can become very complicated. To reduce the complexity, the recently refactored implementation splits the proccess into three separate steps.</simpara> |
| <simpara>Step 1: Collect information</simpara> |
| <simpara>During this phase, members with the name <literal>m2</literal> are looked up in each subscope and collected into an <literal>ComposedMemberInfo</literal> instance by <literal>ComposedMemberInfoBuilder</literal>. |
| The first subscope (left branch in the <xref linkend="fig:unionmemberscope-example"/>) returns an <literal>EObjectDescription</literal> wrapping the <literal>TField m2</literal> of class <literal>C</literal> and hence <literal>TField m2</literal> is added to the <literal>ComposedMemberInfo</literal> instance. The second subscope (right branch in the <xref linkend="fig:unionmemberscope-example"/>) returns a <literal>WrongWriteAccessDescription</literal> wrapping the <literal>TGetter m2</literal> of class <literal>D</literal> and hence <literal>TGetter m2</literal> is added to <literal>ComposedMemberInfo</literal> instance. The reason for <literal>WrongWriteAccessDescription</literal> because <literal>cud.m2</literal> is trying to write to the getter of the same name in <literal>D</literal>.</simpara> |
| <simpara>At the end of this step, two members <literal>public TField m2: int</literal> and <literal>project TGetter m2(): int</literal> are added to <literal>ComposedMemberInfo</literal>.</simpara> |
| <simpara>Step 2: Merge members</simpara> |
| <simpara>This phase merges members of the same name into a composed member based on the information about these members collected in Step 1. Note that merge rules can become quite complicated as many situations must be considered. Sometimes, it is not possible to merge at all. If the merge is possible, we need to consider the following properties, among others,</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>Member kind: what kind of member is the merge result. For instance, what do we get when we merge a field with a setter?</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Type of merge member: What is the return/parameter type of the merge result?</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Accessibility: what is the accessibility of the merge result?</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Optionality/Variadic: Should a parameter of the merge be optional or variadic?</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>The actual merge rules are implemented in the class <literal>UnionMemberFactory</literal> which delegates to either of the classes <literal>UnionMethodFactory</literal>, <literal>UnionFieldFactory</literal>, <literal>UnionGetterFactory</literal> and <literal>UnionSetterFactory</literal>.</simpara> |
| <simpara>In our example, |
| The merge result of <literal>public TField m2: int</literal> and <literal>project TGetter m2(): int</literal> are merged into a <literal>project TGetter m2: int</literal> .</simpara> |
| <simpara>Step 3: Construct the scope entry</simpara> |
| <simpara>In this final step, the actual IEObjectDescription for <literal>m2</literal> is constructed. In our example, since there exists one subscope exposing an <literal>EObjectDescriptionWithError</literal> (here <literal>WrongWriteAccessDescription</literal>), the final result is an instance of <literal>UnionMemberDescriptionWithError</literal>. This error instance is recognized during the linking phase and the error message of the subscope regarding <literal>WrongWriteAccessDescription</literal> is displayed: <emphasis>Union combines fields and getters with name m2 and therefore property m2 is read-only.</emphasis></simpara> |
| <simpara>More details can be found in the API documentation in the code. A good starting point is the class <literal>ComposedMemberScope</literal>.</simpara> |
| </section> |
| <section xml:id="sec:Binding_of_Structurally_References_Types" role="language-n4js"> |
| <title>Structurally References Types</title> |
| <simpara>Scoping of structurally referenced types is similar to binding of members. The structural typing modifier basically filters the members of a type. That is, the structural modifier filters out all non-public members, and the field-only modifier only accept fields. Thus, similar to accessibility aware scoping (<xref linkend="sec:Accessibility_of_types_and_members"/>), the <literal>TypingStrategyAwareMemberScope</literal> encapsulates an original scope and applies these additional filters.</simpara> |
| <simpara>Bindings to additional members of a structurally referenced type is implemented in <literal>MemberScopeProvider.members(ParameterizedTypeRefStructural ..)</literal>. Note that the current implementation does not necessarily bind to the type model (TModule) instance, as these members are part of a type reference. That is, usually these bindings refer to the <link linkend="AC">AST</link> elements. Thus, it is not possible to compare these members directly, instead, a structural comparison has to be applied.</simpara> |
| </section> |
| <section xml:id="sec:Building"> |
| <title>Building</title> |
| <section xml:id="sec:Build_Phases"> |
| <title>Build Phases</title> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Phase 0</simpara> |
| </entry> |
| <entry> |
| <simpara>Loading Resources</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Phase 1: prelinking</simpara> |
| </entry> |
| <entry> |
| <simpara>Create symbols for all resources, includes creation of temporary pre-linked type models</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Phase 2: linking</simpara> |
| </entry> |
| <entry> |
| <simpara>Resolve all links, includes fully-resolved typed models<?asciidoc-br?> |
| includes compilation</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| <simpara>That is, not each resource is loaded, pre-linked and linked separately. Instead, all resources are first loaded, then all resources are pre -inked, and only then all resources are linked.</simpara> |
| </section> |
| <section xml:id="sec:Build_Scenarios"> |
| <title>Build Scenarios</title> |
| <simpara>Consequences:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>do not try to set any types in types builder, only create symbols there (probably not even members of types)</simpara> |
| </listitem> |
| </itemizedlist> |
| </section> |
| <section xml:id="sec:Lazy_linking_problem"> |
| <title>Lazy linking problem</title> |
| <simpara>Lazy linking proxies in the indes may trigger reloading of AST (which leads to invalid disconnected type models):</simpara> |
| <simpara>Lazy links (ending with |x in which x is an index entry of a temporary list used to resolve the link) must not be written into index.</simpara> |
| </section> |
| </section> |
| <section xml:id="sec:Proxies_and_Proxy_Resolution" role="language-n4js"> |
| <title>Proxies and Proxy Resolution (Overview)</title> |
| <simpara>Here we give a brief overview of the different kinds of proxies and when / how they are created and resolved.</simpara> |
| <section xml:id="xtexts-lazy-linking-proxies"> |
| <title>Xtext’s Lazy Linking Proxies</title> |
| <itemizedlist> |
| <listitem> |
| <simpara>URI fragment is <literal>|</literal> <math xmlns="http://www.w3.org/1998/Math/MathML"><mi>n</mi></math> (where <math xmlns="http://www.w3.org/1998/Math/MathML"><mi>n</mi></math> is a non-negative integer).<?asciidoc-br?> |
| <literal>platform:/resource/Project/src/A.n4js#|3</literal></simpara> |
| </listitem> |
| <listitem> |
| <simpara>created by Xtext’s <literal>LazyLinkingResource</literal> in the AST after parsing (they are only ever created in the AST, but the types builder may copy them to the TModule, so they may appear there as well.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>used to represent cross-references defined in the source code (i.e. name of an identifiable element used in source code to refer to that element).<?asciidoc-br?></simpara> |
| <simpara>Since the types builder sometimes copies proxies from AST to TModule (e.g. type of an element that was provided with an explicit type declaration in the source code), these proxies may also appear in the TModule, but only between the types builder phase and the end of the post-processing phase (or later, in case they are unresolvable).</simpara> |
| </listitem> |
| <listitem> |
| <simpara>resolution is handled by <literal>#getEObject(String)</literal> in <literal>LazyLinkingResource</literal>, which recognizes lazy linking URI fragments and then forwards them to <literal>#getEObject(String,Triple)</literal>, which in turn relies on the Xtext infrastructure.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>latest time of resolution: post processing. After post processing has completed, they should all be gone (unless they are unresolvable, e.g. typo in source code).</simpara> |
| </listitem> |
| <listitem> |
| <simpara>fun facts:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>the number after the pipe character is the index of a <literal>Triple</literal> stored in field <literal>proxyInformation</literal> in each <literal>LazyLinkingResource</literal>.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>the resource given before the fragment (e.g. <literal>A.n4js</literal> in the above example) is not the resource the proxy is pointing to (i.e. the resource containing the target EObject), but the resource from where the link originates.</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| </itemizedlist> |
| </section> |
| <section xml:id="standard-emf-proxies"> |
| <title>Standard EMF Proxies</title> |
| <itemizedlist> |
| <listitem> |
| <simpara>URI fragment contains a path to an EObject, using reference names and indices:<?asciidoc-br?> |
| <literal>platform:/resource/Project/src/A.n4js#/1/@topLevelTypes.1/@ownedMembers.0</literal></simpara> |
| </listitem> |
| <listitem> |
| <simpara>created automatically by EMF</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>during deserialization of a TModule <math xmlns="http://www.w3.org/1998/Math/MathML"><mi>M</mi></math> from the Xtext index for all references to a different TModule <math xmlns="http://www.w3.org/1998/Math/MathML"><msup><mi>M</mi><mi>'</mi></msup></math> (see <literal>UserdataMapper</literal>).</simpara> |
| </listitem> |
| <listitem> |
| <simpara>when unloading a resource.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>…​</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara>used to represent</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>cross-references from one TModule to another TModule.</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>astElement</literal> links from TModule to AST whenever the AST is not present (e.g. resource was loaded from Xtext index).</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>definedType</literal> links from AST to TModule after deleting the TModule (this happens in the incremental builder after the pre-linking phase).</simpara> |
| </listitem> |
| <listitem> |
| <simpara>all kinds of links after demand-loading an AST by resolving an <literal>astElement</literal> link (pathological case).</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara>resolution is handled in two ways:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>if the context <literal>EObject</literal> of the proxy, i.e. the one where the proxified cross-reference originates, is contained in an N4JSResource <math xmlns="http://www.w3.org/1998/Math/MathML"><mi>R</mi></math>, then resolution is handled by <literal>N4JSResource#doResolveProxy()</literal> (see also <literal>ProxyResolvingResource</literal> for details).</simpara> |
| <simpara>In this case, special handling is performed to make sure that (a) the target resource is loaded from the index, if possible, and (b) post-processing of the target resource is initiated iff the target resource was loaded from AST (instead from the Xtext index) AND post-processing of resource <math xmlns="http://www.w3.org/1998/Math/MathML"><mi>R</mi></math> is currently in progress or has already been completed.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>otherwise, resolution is handled by standard EMF functionality.</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara>latest time of resolution: none. In fact, some of those proxies (those representing <literal>astElement</literal> links from TModule to AST) must not be resolved at all, because this is not yet properly handled.</simpara> |
| </listitem> |
| </itemizedlist> |
| </section> |
| <section xml:id="_how-is-proxy-resolution-triggered"> |
| <title>How is Proxy Resolution Triggered?</title> |
| <simpara>Resolution of proxies throughout the N4JS implementation is triggered as usually when using EMF, which means: whenever the getter of a EMF cross-reference is invoked and the value is still a proxy, the EMF-generated code of the getter will automatically trigger resolution of this proxy. For details look at the EMF-generated code of the getter of any cross-reference (<literal>IdentifierRefImpl#getId()</literal> would be a good example).</simpara> |
| </section> |
| <section xml:id="_when-is-proxy-resolution-allowed"> |
| <title>When is Proxy Resolution Allowed?</title> |
| <simpara>So, at what time is it legal to trigger such a proxy resolution? Or, more concretely, during which resource load states (<xref linkend="sec:N4JS_Resource_Load_States"/>) is it legal to trigger proxy resolution? In fact, asking the question in this way is incorrect or at least not very helpful, because the answer would be (almost) always. The better question is: which components of the system / which parts of the code base are allowed to trigger proxy resolution?</simpara> |
| <simpara>For example, triggering resolution is disallowed in the <literal>ASTStructureValidator</literal> and <literal>N4JSTypesBuilder</literal>, but for the outside client code such as a JUnit test it is allowed to trigger proxy resolution as early as right after parsing. For an example of the latter see test <literal>#testStateFullyProcessed_triggeredOnlyThroughProxyResolution()</literal> in <literal>N4JSResourceLoadStatesTest</literal>.</simpara> |
| <simpara>In summary, we can state the rule that the <emphasis role="strong">internal N4JS implementation</emphasis> must not trigger any proxy resolution until installation of the derived state has completed, i.e. not before resource load state "Fully Initialized", but <emphasis role="strong">client code</emphasis> may trigger proxy resolution as early as right after parsing, i.e. already in resource load state "Loaded".</simpara> |
| </section> |
| </section> |
| </chapter> |
| <chapter xml:id="_validation"> |
| <title>Validation</title> |
| <warning> |
| <simpara>This chapter may be outdated.</simpara> |
| </warning> |
| <simpara>In this chapter the concept for the validation infrastructure of the N4JS IDE is described.</simpara> |
| <simpara>Requirements:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>like in old IDE centralize issue codes and messages</simpara> |
| </listitem> |
| <listitem> |
| <simpara>instead of the DLTK API, the Xtext Validation API should be used</simpara> |
| </listitem> |
| <listitem> |
| <simpara>there should be an unified approach / call to produce validation messages (as in old IDE with the call of reportProblem)</simpara> |
| </listitem> |
| </itemizedlist> |
| <section xml:id="sec:validation_overview" role="language-n4js"> |
| <title>Validation overview</title> |
| <itemizedlist> |
| <listitem> |
| <simpara>in Xtext in most cases validation should triggered after all Xtext resources are linked (so everything is resolved), so most validations are defined in N4JSValidator and there in composed validators</simpara> |
| </listitem> |
| <listitem> |
| <simpara>in Xtext there are resource diagnostics and validation diagnostics</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>resource diagnostics are produced for issues related to found syntax errors and linking errors</simpara> |
| </listitem> |
| <listitem> |
| <simpara>validation diagnostics are produced for issues found during semantic validation (model invariants)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Note, that you can only produce diagnostics only for the resource currenlty validated - e.g. it isn’t possible to create a marker for a duplicate issue in the other resource that contains the first element while producing the issue for the second - you have to create the diagnostic when validating the other resource</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara>validation message sources</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>parser, e.g. ANTLR parser: messages are contained and produced internally by the parser but can be adapted by a customized SyntaxErrorMessageProvider (see link section below)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>lazy linking - N4JSLinker (extension of lazy linker): custom handling exceptions from value converters, that produces own issues as XtextLinkingDiagnostic (this is a resource diagnostic)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>lazy linking - ASTStructureValidator: traverses AST and produces custom DiagnosticMessage (triggered during N4JSLinker.doLinkModel), creates XtextLinkingDiagnostic (this is a resource diagnostic). The ASTStructureValidator checks for things like allowed occurence of continue, break, return as well as allowed labels</simpara> |
| </listitem> |
| <listitem> |
| <simpara>linking - default: uses the ILinkingDiagnosticMessageProvider to create error messages for typical error cases (referenced not resolved, unique constraint violation, bounds contraints violation, etc.)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>linking - ErrorAwareLinkingService: In N4JS we have introduced a custom IEObjectDescription AbstractDescriptionWithError that holds issue code and error message and ErrorAwareLinkingService as extension of DefaultLinkingService to produce for every AbstractDescriptionWithError a XtextLinkingDiagnostic (this is a resource diagnostic) - the usual Xtext behavior would be to produce a linking error with linking disabled, with our customization linking still works but the error message/marker is produced as well. Currently there are these custom AbstractDescriptionWithError implementations, that are produced during scoping</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>AmbiguousImportDescription: indicate an ambiguous wildcard import</simpara> |
| </listitem> |
| <listitem> |
| <simpara>PlainAccessOfAliasedImportDescription: indicate accessing type directly instead of using the alias it has been imported with</simpara> |
| </listitem> |
| <listitem> |
| <simpara>InvisibleMemberDescription: indicate accessing a member, that is not visible to the caller due to access modifier restrictions</simpara> |
| </listitem> |
| <listitem> |
| <simpara>InvisibleTypeOrVariableDescription: indicate accessing a type / variable or function, that is not visible to the caller due to access modifier restrictions</simpara> |
| </listitem> |
| <listitem> |
| <simpara>WrongFieldAccessorDescription: setter/getter access used in wrong context</simpara> |
| </listitem> |
| <listitem> |
| <simpara>WrongStaticAccessorDescription: static/non-static access used in wrong context</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara>N4JSTypeSystem:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>error: cannot type given expression</simpara> |
| </listitem> |
| <listitem> |
| <simpara>error: element is not a subtype of another type</simpara> |
| </listitem> |
| <listitem> |
| <simpara>RuleFailedException is thrown in type system, then handled as result where out of the contained information the AbstractDeclarativeValidator API is used to create issues via calls to error(..) or warning(..) methods</simpara> |
| </listitem> |
| <listitem> |
| <simpara>RuleFailedExceptionWithoutStacktrace - sub class of RuleFailedException</simpara> |
| </listitem> |
| <listitem> |
| <simpara>N4JSTypeValidator.createError → delegates to AbstractDeclarativeValidator, or directly calls AbstractDeclarativeValidator.error</simpara> |
| </listitem> |
| <listitem> |
| <simpara>XsemanticsValidatorErrorGenerator (currently not used)</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara>N4JSValidation: composed validators, declarative checks, i.e. using annotation @Check at methods, issues mostly created by using combination of message, context object, EFeature and index passed to AbstractDeclarativeValidator methods like error(..), warning(..)</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>N4JSClassValidator - checks related to classes, like duplicate member check</simpara> |
| </listitem> |
| <listitem> |
| <simpara>N4JSFunctionValidator - checks for function expressions in expression statements and creates an error, but this validation is already done by the ASTStructureValidator</simpara> |
| </listitem> |
| <listitem> |
| <simpara>N4JSImportValidator - checks invalid, duplicated and invalid imports</simpara> |
| </listitem> |
| <listitem> |
| <simpara>N4JSInterfaceValidator - currently checks only for static members</simpara> |
| </listitem> |
| <listitem> |
| <simpara>N4JSRoleValidator - currently checks only for static members</simpara> |
| </listitem> |
| <listitem> |
| <simpara>N4JSStrictValidator - checks for function expressions in expression statements and creates an error, but this validation is already done by the ASTStructureValidator, thus this composite test is commented out</simpara> |
| </listitem> |
| <listitem> |
| <simpara>N4JSTypeValidator - this is type system, see above</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara>Check types and check modes</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>Check types are used in <literal>@Check</literal> annotation to influence, when a validation is triggered</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><literal>FAST</literal> - Check is declared as fast</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>NORMAL</literal> - the common checks</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>EXPENSIVE</literal> - long running checks</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara>Check modes decide using check types when to run which types of checks</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><literal>FAST_ONLY</literal> - used by Xtext editor to execute FAST checks on every key stroke and in embedded Xtext editors</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>NORMAL_ONLY</literal> - not used in Xtext framework itself</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>EXPENSIVE_ONLY</literal> - not used in Xtext framework itself</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>NORMAL_AND_FAST</literal> - used by Xtext editor to execute FAST and NORMAL checks on file save as well as on marker update for changed resources (delta calculated by Xtext builder)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>ALL - executes all types of checks when invoking explicitley the Validate action in the context menu of the Xtext editor</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara>So by default check types and modes aren’t configurable at runtime</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara>Severity</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>types</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><literal>ERROR</literal> - indicates a serious issue, that later will prevent things like builder, compiler and so on, to run</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>WARNING</literal> - indicates a possible semantic problem, where the user have to decide how to handle it. Issues with such severity won’t stop any post processing</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>INFO</literal> - only an information hint for the user. Note, it is not allowed to create a diagnostic with severity INFO.</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara>can be statically provided by calling the AbstractDeclarativeValidator methods error(..), warning(..), info(..) or directly pass the severity to a sub class of AbstractValidationDiagnostic (e.g. FeatureBasedDiagnostic, RangeBasedDiagnostic)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>can also determined dynamically at runtime with using the IssueSeveritiesProvider and a implementation of IPreferenceValuesProvider (e.g. the EclipsePreferencesProvider that uses the Eclipse preference store and preference page)</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara>Issue codes</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>used to identify an issue elsewhere, e.g. when applying an quickfix for a validation issue but also for configuring validation handling (e.g. in a Eclipse preference page).</simpara> |
| </listitem> |
| <listitem> |
| <simpara>We can use this issue code to also externalize the issue messages at a central place</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara>Issue data: String array to store additional data to be used in other places (e.g. hints for quickfixes)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>message: The message shown as text for the marker created at the resource in Eclipse and shown in the Xtext editor but also available by the methods getWarnings and getErrors at the XtextResource itself and so usable when logging messages to console in headless mode</simpara> |
| </listitem> |
| </itemizedlist> |
| </section> |
| <section xml:id="sec:validation_control_flow" role="language-n4js"> |
| <title>Validation control flow</title> |
| <simpara><xref linkend="fig:cd_validation"/> gives an overview over the common control flow that triggers validation.</simpara> |
| <figure xml:id="fig:cd_validation" role="center"> |
| <title>Validation control flow</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/10_validation/images/cd_validation.svg"/> |
| </imageobject> |
| <textobject><phrase>cd validation</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara>Validation is either triggered by dirty state handling (editing an Xtext document without saving starts a validation job) or by the automated build (invoked directly or started by resource changes in the project e.g. after saving a file).</simpara> |
| <simpara>While in dirty state handling the current resource is already parsed and resolved the builder have to load the resource.</simpara> |
| <simpara>All issues collected during load (i.e. during parsing, linking and scoping) the resource are added to the resource.</simpara> |
| <simpara>In the automated build process there is step <literal>updateMarkers</literal> that triggers the validation.</simpara> |
| <simpara>The <literal>SourceContainerAwareResourceValidator</literal> is a customization by us to handle only files that are contained in folders declared as source container by the package.json file.</simpara> |
| <simpara>The <literal>CancelableDiagnostican</literal>, called by the resource validator, iterates over all elements contained in the resource. For each element the bound validator is called, in our case <literal>N4JSValidator</literal>, as it is registered as validator for the N4JS EPackage (in <literal>AbstractN4JSValidator</literal>).</simpara> |
| <simpara>As this validator extends <literal>AbstractDeclarativeValidator</literal> in the first step all methods that are annotated with @Check and that have exactly one parameter are collected keyed by the type of their input parameter. The result of this collection process is cached. There is a defined order how the methods are collected:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>all local methods</simpara> |
| </listitem> |
| <listitem> |
| <simpara>all methods recursively found in the super classes of the current class</simpara> |
| </listitem> |
| <listitem> |
| <simpara>all methods found for the in the composed check annotation defined validators (by applying this algorithm as well)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>all methods recursively found in the composed checks in the super classes of the current class (by applying this algorithm as well)</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>The <literal>N4JSValidator</literal> filters all methods that uses the type of the currently traversed element from the before collected check methods and invokes them with the element from the resource.</simpara> |
| <simpara>The Xsemantics type system validator is used as one of the composed validators in <literal>N4JSValidator</literal>. So although <literal>N4JSValidator</literal> extends <literal>N4JSTypeSystemValidator</literal>, <literal>N4JSTypeSystemValidator</literal> just re-uses the validation infrastructure but not its call hierarchy.</simpara> |
| </section> |
| <section xml:id="sec:validation_issue_ids" role="language-n4js"> |
| <title>Issue IDs and Messages</title> |
| <simpara>For now the NLS validation message bundle resides in<?asciidoc-br?> |
| <literal>/org.eclipse.n4js/src/org/eclipse/n4js/validation/messages.properties</literal><?asciidoc-br?> |
| The entries in the messages.properties follows the pattern as described in <literal>NLSProcessor</literal>, the NLS class is <literal>IssueCodes</literal></simpara> |
| <tip> |
| <simpara>We use the same pattern for semver and json.</simpara> |
| </tip> |
| <itemizedlist> |
| <listitem> |
| <simpara>IDs shouldn’t be to long, as there might be a lot of markers and the issue codes are stored in memory</simpara> |
| </listitem> |
| <listitem> |
| <simpara>the ID should encode where the issue has been created, therefore there should be common used prefixes like</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>PRS for parser (not used yet)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>VCO for value converter</simpara> |
| </listitem> |
| <listitem> |
| <simpara>AST for issues found during AST traversal</simpara> |
| </listitem> |
| <listitem> |
| <simpara>LIN for issues found during scoping/linking (not used yet)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>TYS for type system</simpara> |
| </listitem> |
| <listitem> |
| <simpara>VAL for semantic validation (not used yet)</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara>besides the source also the domain of the issue should be encoded (the following list may reduced or extended over time, when it gets obvious which sorts of domain specific validations are required in which frequency)</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>CLF for issues common to all classifiers</simpara> |
| </listitem> |
| <listitem> |
| <simpara>CLA for classes (not used yet)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>ROL for roles</simpara> |
| </listitem> |
| <listitem> |
| <simpara>FUN for function</simpara> |
| </listitem> |
| <listitem> |
| <simpara>IMP for imports</simpara> |
| </listitem> |
| <listitem> |
| <simpara>VAR for variables (not used yet)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>MEM for classifier members in general</simpara> |
| </listitem> |
| <listitem> |
| <simpara>OLI for object literals (not used yet)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>ENU for enumerations (not used yet)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>ARR for array literals (not used yet)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>ANN for annotation related issues (not used yet)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>EXP for expression related issues (assignment expression, binary expression, etc.) (not used yet)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>STMT for statement related issues, such as if-else (conditional) , loops, switch etc.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>PRP for property access related issues (not used yet)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>EXC for exception handling related issues (not used yet)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>LBL for labels related issues (not used yet)</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara>also technical validation aspects can be encoded</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>DUP for duplicate checks</simpara> |
| </listitem> |
| <listitem> |
| <simpara>VIS for visibility checks (public, private, export, etc.)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>STR for issues related only applied in strict mode</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara>examples</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><literal>IMP_AMBIGUOUS</literal></simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>VIS_ILLEGAL_MEMBER_ACCESS</literal></simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>CLF_ABSTRACT_FINAL</literal></simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>AST_RESERVED_IDENTIFIER</literal></simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>VCO_HEXINT_CONVERT_EMPTY_STR</literal></simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>TYS_NO_SUBTYPE</literal></simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| </itemizedlist> |
| </section> |
| <section xml:id="sec:validation_usage_patterns" role="language-n4js"> |
| <title>Usage Pattern</title> |
| <simpara>Due to the different places and circumstances a real unification of the API wasn’t possible yet (and wouldn’t have made sense), so there are these different usage patterns</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>in a custom error aware EObjectDescription like WrongFieldAccessorDescription you just return the issue code in getIssueCode and the message created using the issue code as well as the replacements for the wildcards in getMessage</simpara> |
| </listitem> |
| <listitem> |
| <simpara>in a validator extending AbstractDeclarativeValidator you just call <literal>addIssue(message, context, EFeature, issueCode)</literal>. The message you have to create before by calling the corresponding <literal>getMessageFor[ISSUE_ID]</literal> method passing the required wildcard replacement</simpara> |
| </listitem> |
| <listitem> |
| <simpara>in the ASTStructureValidator you have to call <literal>producer.addDiagnostic(new DiagnosticMessage(IssueCodes.messageFor[ISSUE_ID](wildcard1, wildcard2, ..), IssueCodes.getDefaultSeverity(IssueCodes.[ISSUE_ID]), IssueCodes.[ISSUE_ID]))</literal></simpara> |
| </listitem> |
| <listitem> |
| <simpara>in the custom value converters you have to pass the information to an exception, so the call is: <literal>new N4JSValueConverterException(IssueCodes.getMessageFor[ISSUE_ID](wildcard1, wildcard2, ..), IssueCodes.[ISSUE_ID], node, null)</literal>. Beside this exception also N4JSValueConverterWithValueException is used in some places. In N4JSLinker then these exceptions are catched and a DiagnosticMessage is created out of the informations contained in these exceptions.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>As Xsemantics uses hard wired error or warning in its grammar you cannot adapt these places, but there are currently only three messages produced by Xsemantic (cannot type, not a sub type, null object passed to system). They are all handled in N4JSTypeValidator.createError where the message from Xsemantic is split up in its parts and then passed as wild card replacements to e.g. <literal>IssueCodes.getMessageForTYS_NO_SUBTYPE</literal>.</simpara> |
| </listitem> |
| </itemizedlist> |
| </section> |
| <section xml:id="sec:validation_links"> |
| <title>Links</title> |
| <itemizedlist> |
| <listitem> |
| <simpara><link xl:href="http://www.eclipse.org/Xtext/documentation.html#validation">Xtext Runtime Concepts: Validation</link></simpara> |
| </listitem> |
| <listitem> |
| <simpara><link xl:href="http://blog.dietmar-stoll.de/2013/04/multiple-validators-in-xtext.html">Multiple validators in Xtext</link></simpara> |
| </listitem> |
| <listitem> |
| <simpara>Customize error messages</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><link xl:href="http://zarnekow.blogspot.de/2010/06/customizing-error-messages-in-xtext-10.html">Customizing error messages in Xtext</link></simpara> |
| </listitem> |
| <listitem> |
| <simpara><link xl:href="http://blog.dietmar-stoll.de/2012/07/custom-syntax-error-messages-with-quick.html">Custom syntax error messages with quickfix</link></simpara> |
| </listitem> |
| <listitem> |
| <simpara><link xl:href="http://stackoverflow.com/questions/14526524/xtext-customizing-error-msg-by-unordered-groups">Xtext: customizing error messages by unordered groups</link></simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| </itemizedlist> |
| </section> |
| </chapter> |
| <chapter xml:id="_references"> |
| <title>References</title> |
| <warning> |
| <simpara>This chapter maybe outdated.</simpara> |
| </warning> |
| <section xml:id="sec:usecases"> |
| <title>Use cases</title> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Compilation</simpara> |
| </entry> |
| <entry> |
| <simpara>for deciding in incremental builder which resources requires a recompilation</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Editing</simpara> |
| </entry> |
| <entry> |
| <simpara>Dirty state calculation: for deciding which resources needs to be reparsed as references have changed</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>UI</simpara> |
| </entry> |
| <entry> |
| <simpara>Such as <keycap>Find references</keycap>, find all places in the workspaces that points to the selected element</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Tools</simpara> |
| </entry> |
| <entry> |
| <simpara>requiring references, such as refactorings, e.g., rename refactoring: apply the renaming of the element also to all references to it (like find by references)</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section xml:id="sec:calculation_algorithm" role="language-n4js"> |
| <title>Calculation algorithm</title> |
| <section xml:id="sec:Xtext_default_implementation"> |
| <title>Xtext default implementation</title> |
| <simpara>Using Reference Descriptions:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>default implementation contained in method <literal>createReferenceDescriptions</literal> of <literal>o.e.x.resource.impl.DefaultResourceDescriptionStrategy</literal></simpara> |
| </listitem> |
| <listitem> |
| <simpara>iterates over all EReferences of the EClass of the current element</simpara> |
| </listitem> |
| <listitem> |
| <simpara>navigates all references and resolves them (already done before in DefaultResourceDescription.computeReferenceDescriptions)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>create reference description objects for all these references</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>In case of N4JS and the types, reference descriptions are also created for references to types model elements (definedType) and for references from Types element to AST.</simpara> |
| <simpara>We do not use this default implementation for two reasons:</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>expensive</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Default implementation of reference descriptions only calculates the direct dependencies but not the transitive ones + the calculation of the URIs is very expensive.</simpara> |
| </listitem> |
| </orderedlist> |
| </section> |
| <section xml:id="sec:N4_implementation"> |
| <title>N4JS implementation</title> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>the Xtext default implementation is disabled by let <literal>N4JSResourceDescription.computeReferenceDescriptions</literal> return an empty list. Also the bound <literal>N4JSDescriptionUtils</literal> returns an empty list for collectOutgoingReferences</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Calculating <emphasis>direct references</emphasis> are only done inside <literal>N4JSResourceDescription.getImportedNames</literal> (that uses newly introduced <literal>N4JSCrossReferenceComputer.computeCrossRefs</literal> for collecting all direct dependencies) - here only (parameterized) type refs, types and identifiable elements are collected</simpara> |
| </listitem> |
| <listitem> |
| <simpara>collect all <emphasis>transitive dependencies</emphasis>, i.e. all super classes, consumed roles and implemented interfaces in the type hierarchy and add their resources as dependency (this is done in <literal>N4JSResourceDescription.getImportedNames</literal> (after calculating all direct dependencies with <literal>N4JSResourceDescriptionStrategy</literal>)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>bind <literal>N4JSReferenceQueryExecutor</literal> as a custom implementation to calculate the target URIs for an given target element and bind <literal>N4JSReferenceFinder</literal> as a custom implementation to calculate reference descriptions to be used by the default Xtext found references UI (use case UI)</simpara> |
| </listitem> |
| </orderedlist> |
| <simpara><literal>ClusteringBuilderState.doUpdate</literal> calculates if a dependent resource has changed (in the context of calculating the <literal>DefaultResourceDescriptionDelta</literal> out of the old and new resource descriptions). Each resource description consists of <literal>EObjectDescriptions</literal>. The <literal>EObjectDescription</literal> for <literal>Script</literal> also contains the types model (<literal>TModule</literal>) for the resource. The references between resources are implemented via the type model. If it has changed (compared with the user data of the old <literal>EObjectDescription</literal>) then all other resource descriptions registered as been dependent on the resource (the qualified names of the resource descriptions are serialized as imported names within the resource description) will be notified that a reparse is needed.</simpara> |
| <simpara>For <emphasis>dirty state</emphasis> the same behavior is achieved via the dirty state editor support using the resource set of the editor (instead the file system resources).</simpara> |
| <simpara>As the primitive and built-in types are fixed they are ignored when calculating the dirty state. When calculating dependending resources and dirty state the reference of an AST element to its defining type is ignored as is the reference from the type to its AST element</simpara> |
| <simpara><link linkend="fig:cd_classes">Classes</link> shows the different entry points (user actions) and classes involved in the process.</simpara> |
| <informalfigure xml:id="fig:cd_classes" role="center"> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/11_references/images/cd_classes.svg"/> |
| </imageobject> |
| <textobject><phrase>title-"Reference handling"</phrase></textobject> |
| </mediaobject> |
| </informalfigure> |
| </section> |
| </section> |
| <section xml:id="sec:PerformanceOfDependencyCalculation" role="language-n4js"> |
| <title>Performance Of Dependency Calculation</title> |
| <simpara>Concerning frequency and contexts it is clear, that triggering <literal>Find references</literal> and rename refactoring is not as frequent as editing (n4)js files that leads to dirty state (very often as happens when editing) and to trigger compilation (at file save, also often). Calculating if a resource is affected by a change (dirty state calculation) may not be too expensive. But running compilation for too many files (or the wrong set of files) due to incorrect dirty state calculation is expensive.</simpara> |
| <simpara><literal>N4JSResourceDescription.getImportedNames</literal> is invoked on every edit of a file in the editor, so on every edit the complete content has to be retraversed for possible new references (expensive but not avoidable). For the types of all found references the super types have to recalculated. Traversing the type hierarchy shouldn’t be too expensive.</simpara> |
| <simpara>Possible optimization could be:</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>caching of traversing referenced types whose resources had not changed since last edit</simpara> |
| </listitem> |
| <listitem> |
| <simpara>not traversing types that are imported but non of their features are used within the current resource</simpara> |
| </listitem> |
| </orderedlist> |
| <simpara><emphasis>Those optimization should be done only if there are real performance issues are discovered.</emphasis></simpara> |
| </section> |
| <section xml:id="sec:kinds_of_references" role="language-n4js"> |
| <title>Kinds of references</title> |
| <section xml:id="sec:Cross_References_to_be_ignored"> |
| <title>Cross References to be ignored</title> |
| <informaltable frame="all" rowsep="1" colsep="1"> |
| <tgroup cols="3"> |
| <colspec colname="col_1" colwidth="33.3333*"/> |
| <colspec colname="col_2" colwidth="33.3333*"/> |
| <colspec colname="col_3" colwidth="33.3334*"/> |
| <thead> |
| <row> |
| <entry align="left" valign="top">Element</entry> |
| <entry align="left" valign="top">Reference</entry> |
| <entry align="left" valign="top">Explanation</entry> |
| </row> |
| </thead> |
| <tbody> |
| <row> |
| <entry align="left" valign="top"><simpara>TypeDefiningElement</simpara></entry> |
| <entry align="left" valign="top"><simpara>definedType: Type</simpara></entry> |
| <entry align="left" valign="top"><simpara>always inner resource change (e.g., Functions, Classifier)</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>types::Type</simpara></entry> |
| <entry align="left" valign="top"><simpara>astElement</simpara></entry> |
| <entry align="left" valign="top"><simpara>as always inner resource change</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>ImportDeclaration</simpara></entry> |
| <entry align="left" valign="top"><simpara>importedModule: TModule</simpara></entry> |
| <entry align="left" valign="top"><simpara>only affected if the resource name changes (and such a change cannot be performed dirty)</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>ContinueStatement</simpara></entry> |
| <entry align="left" valign="top"><simpara>label: LabelledStatement</simpara></entry> |
| <entry align="left" valign="top"><simpara>always inner resource changes</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>BreakStatement</simpara></entry> |
| <entry align="left" valign="top"><simpara>label: LabelledStatement</simpara></entry> |
| <entry align="left" valign="top"><simpara>always inner resource changes</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>types::PrimitiveType</simpara></entry> |
| <entry align="left" valign="top"><simpara>autocoercedObject: TClassifier</simpara></entry> |
| <entry align="left" valign="top"><simpara>fixed (immutable) and internal</simpara></entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section xml:id="sec:Cross_References_to_be_handled" role="language-n4js"> |
| <title>Cross References to be handled</title> |
| <simpara>Cross References</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>types::ParameterizedTypeRef → declaredType : Type</simpara> |
| </listitem> |
| <listitem> |
| <simpara>* → N4GetterDeclaration: N4FieldDeclaration type, but references to getter can be done also done from outside</simpara> |
| </listitem> |
| <listitem> |
| <simpara>* → N4SetterDeclaration: undef, references to setter can be done also done from outside</simpara> |
| </listitem> |
| <listitem> |
| <simpara>types::PrototypeTypeRef → type : Type</simpara> |
| </listitem> |
| </orderedlist> |
| <simpara>Contained <literal>ParameterizedTypeRef</literal> and <literal>TypeVariables</literal> (that internally references to <literal>Type</literal>):</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>references to declared super types of a type (Class, Role, Interface), i.e. superType, consumedRoles, implementedInterfaces</simpara> |
| </listitem> |
| <listitem> |
| <simpara>TypeVariable → declaredUpperBounds</simpara> |
| </listitem> |
| <listitem> |
| <simpara>References in type arguments:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>Wildcards → upper and lower bounds, e.g. <literal>var List<? super A> l1;</literal></simpara> |
| </listitem> |
| <listitem> |
| <simpara>direct type references, e.g. <literal>var List<A> l;</literal></simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| </orderedlist> |
| <simpara>Cross References to IdentifiableElement (types):</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>IdentifierRef → id : IdentifiableElement</simpara> |
| </listitem> |
| <listitem> |
| <simpara>NamedImportSpecifier → importedElement : IdentifiableElement</simpara> |
| </listitem> |
| <listitem> |
| <simpara>ParameterizedPropertyAccessExpression → property : IdentifiableElement</simpara> |
| </listitem> |
| <listitem> |
| <simpara>PropertyAccessExpression → property : IdentifiableElement</simpara> |
| </listitem> |
| </orderedlist> |
| <simpara>Contained IdentifierRef (that internally references to IdentifiableElement):</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>ParameterizedCallExpression → target</simpara> |
| </listitem> |
| <listitem> |
| <simpara>as PrimaryExpression in MemberExpression</simpara> |
| </listitem> |
| </orderedlist> |
| </section> |
| </section> |
| <section xml:id="sec:transitive_dependencies"> |
| <title>Transitive dependencies</title> |
| <simpara>Besides the direct dependencies we also need the transitive dependencies, as demonstrated in the following example.</simpara> |
| <formalpara xml:id="ex:transdepex"> |
| <title>Transitive Dependency</title> |
| <para> |
| <programlisting language="java" linenumbering="unnumbered">export class A { |
| public myMethod() |
| }</programlisting> |
| </para> |
| </formalpara> |
| <formalpara> |
| <title>Transitive Dependency pt.2</title> |
| <para> |
| <programlisting language="java" linenumbering="unnumbered">export class B extends my/test/A { |
| }</programlisting> |
| </para> |
| </formalpara> |
| <formalpara> |
| <title>Transitive Dependency pt.3</title> |
| <para> |
| <programlisting language="java" linenumbering="unnumbered">export class C extends my/test/B { |
| myMethodC() { |
| this.myMethod() |
| } |
| }</programlisting> |
| </para> |
| </formalpara> |
| <simpara>If the name of <literal>myMethod</literal> in A changes, C should get dirty. This can get more complicated, if, e.g., a method in a consumed role is renamed, which then leads to binding references to super types.</simpara> |
| <simpara>Therefore all direct and indirect super types are calculated (including super classes, consumed roles and implemented interfaces) for all found directly referenced types. The qualified names of their resources are added to the list of imported names. <footnote><simpara>One could think of an optimization to only register those types that are not just imported or declared, but whose features are really in use. E.g., in one file another type be imported (and even used as type of variable), but non of its member is used. So changes to these members wouldn’t affect the current resources. However this might miss certain cases. E.g., when a method in the super class is removed and now the method with same signature of a consumed role would be used. The method of the role has no been used before, yet must not be ignored. Thus, currently all super classes, roles and interfaces and referenced classes are added as dependency regardless if their members are called.</simpara></footnote></simpara> |
| <simpara>Other transitive dependencies:</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>call of member mixed by a consumed role</simpara> |
| <orderedlist numeration="loweralpha"> |
| <listitem> |
| <simpara>role is consumed by role consumed by this class</simpara> |
| </listitem> |
| <listitem> |
| <simpara>role is consumed by class inherited by this class</simpara> |
| </listitem> |
| </orderedlist> |
| </listitem> |
| <listitem> |
| <simpara>call of member available by implemented interface</simpara> |
| <orderedlist numeration="loweralpha"> |
| <listitem> |
| <simpara>interface is implemented by role consumed by this class</simpara> |
| </listitem> |
| <listitem> |
| <simpara>interface is implemented by class inherited by this class</simpara> |
| </listitem> |
| </orderedlist> |
| </listitem> |
| <listitem> |
| <simpara>call of member available by extended class</simpara> |
| <orderedlist numeration="loweralpha"> |
| <listitem> |
| <simpara>class is extended by class inherited by this class</simpara> |
| </listitem> |
| </orderedlist> |
| </listitem> |
| <listitem> |
| <simpara>chained method calls</simpara> |
| <orderedlist numeration="loweralpha"> |
| <listitem> |
| <simpara>method is of type that itself has members which are directly called, so the type is not directly imported or referenced by name in the caller but indirectly required</simpara> |
| </listitem> |
| <listitem> |
| <simpara>method is of type that itself inherits members which are directly called, so the type (and its super types) is not directly imported or referenced by name in the caller but indirectly required</simpara> |
| </listitem> |
| </orderedlist> |
| </listitem> |
| </orderedlist> |
| <informalexample> |
| <simpara>Each type is defined in its own file.</simpara> |
| <programlisting language="java" linenumbering="unnumbered">export class MyClassOne { |
| myMethodOne() { |
| var MyClassTwo instance; |
| instance.getElement().myMethodThree() |
| } |
| }</programlisting> |
| <programlisting language="java" linenumbering="unnumbered">export class MyClassTwo { |
| MyClassThree getElement() { |
| return new MyClassThree; |
| } |
| }</programlisting> |
| <programlisting language="java" linenumbering="unnumbered">export class MyClassThree { |
| void myMethodThree() {} |
| }</programlisting> |
| <simpara>If <literal>myMethodThree</literal> is renamed this should affect MyClassOne.</simpara> |
| <simpara>Note that the method call in <literal>MyClassOne</literal> directly binds to the method in <literal>MyClassThree</literal>. However, the dependencies are only managed by means of types. So, from that perspective, the dependency between <literal>MyClassOne</literal> and <literal>MyClassThree</literal> is indirect.</simpara> |
| <programlisting language="java" linenumbering="unnumbered">export class MyClassOne { |
| void myMethodOne() { |
| var MyClassTwo instance; |
| instance.myMethodTwo().getElement().myMethodFour() |
| } |
| }</programlisting> |
| <programlisting language="java" linenumbering="unnumbered">export class MyClassTwo { |
| MyClassThree<MyClassFour> myMethodTwo() { |
| return null; |
| } |
| }</programlisting> |
| <programlisting language="java" linenumbering="unnumbered">export class MyClassThree<T extends MyClassFour> { |
| T element; |
| |
| T getElement() { |
| return this.element; |
| } |
| }</programlisting> |
| <programlisting language="java" linenumbering="unnumbered">export class MyClassFour { |
| void myMethodFour() { |
| } |
| }</programlisting> |
| <simpara>If <literal>myMethodFour</literal> is renamed this should affect <literal>MyClassOne</literal>.</simpara> |
| </informalexample> |
| <simpara>More examples are found in the tests (cf. <literal>..ide.n4js.dirtystate.BuilderParticipantPluginTest</literal> and <literal>…​BuilderParticipantPluginUITest</literal>)</simpara> |
| </section> |
| <section xml:id="sec:find-references" role="language-n4js"> |
| <title>Find references</title> |
| <simpara>Find references is perceived as a feature in Eclipse IDE, but its implementation can also be useful in a headless scenario, e.g. in the compiler to drop dead code. |
| Therefore, as opposed to the Xtext default implementations, the code was refactored to split the parts that depend on the UI from the non-UI dependent logic (see <literal>org.eclipse.n4js.findReferences</literal> vs. <literal>org.eclipse.n4js.ui.search</literal>).</simpara> |
| <section xml:id="_background"> |
| <title>Background</title> |
| <simpara>Since no reference descriptions are stored in the index for N4JS resources, the cross references have to be found by other means. |
| That is, the list of imported names is used as an indicator to find resources that have a potential dependency to the searched element. |
| These resources have to be checked thoroughly. |
| That is, their clear text representation is checked at a first step against the clear text representation of the found element before the resource is fully loaded and cross references are resolved.</simpara> |
| <simpara>The decision to drop reference descriptions from the index was deliberate since they would only report bogus information in the context of inheritance, e.g. a method <literal>getA</literal> of type <literal>B</literal> my be overridden by <literal>getA</literal> in type <literal>C</literal>. |
| Concrete bindings against <literal>C.getA</literal> should also be reported as references to <literal>B.getA</literal> since they identify the same public API of the type hiearchy around <literal>B</literal>. |
| Therefore reference descriptions could not be used to find dependencies between source snippets.</simpara> |
| </section> |
| <section xml:id="_how-find-references-work"> |
| <title>How Find References Work</title> |
| <simpara>Methods for finding references are provided Xtext’s interface <literal>IReferenceFinder</literal> and can be used both by the UI or headlessly. |
| The N4JS implementation of this interface for the N4JS language is the class <literal>ConcreteSyntaxAwareReferenceFinder</literal>. |
| One of the key methods defined by the <literal>IReferenceFinder</literal> is <literal>void findAllReferences(TargetURIs, IResourceAccess, IResourceDescriptions, Acceptor, IProgressMonitor)</literal> that finds all places in all resources of the index whereby those places cross-reference one of the URIs contained in <literal>TargetURIs</literal> .</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><literal>TargetURIs</literal> contains the set of URIs to be searched. The caller of <literal>IReferenceFinder</literal> is responsible for collecting the <literal>Target URIs</literal> to be searched.</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>IResourceAccess</literal> is used to search local references. This is needed because local references are usually not index.</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>IResourceDescriptions</literal> is the indexed.</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>Acceptor</literal> is called when a reference is found.</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>IProgressMonitor</literal> is used for showing progress bar (can be null).</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>In the following, we will have a look at the workflow to find references when triggered in the UI. |
| After understanding the UI case, the workflow of find references in the headless case should be self-explanatory.</simpara> |
| <simpara><link linkend="fig:findreference_workflow">Find reference workflow</link> shows the workflow of find references when triggered in the UI.</simpara> |
| <informalfigure xml:id="fig:findreference_workflow" role="center"> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/11_references/images/findreference_workflow.svg"/> |
| </imageobject> |
| <textobject><phrase>title-"Find reference workflow"</phrase></textobject> |
| </mediaobject> |
| </informalfigure> |
| <simpara>The following example will be used for explanation.</simpara> |
| <formalpara> |
| <title>A.n4js</title> |
| <para> |
| <programlisting language="n4js" linenumbering="unnumbered">import {B} from "B"; |
| let b = new B(); // B here is an IdentifierRef referring to TClass B in B.n4js</programlisting> |
| </para> |
| </formalpara> |
| <formalpara> |
| <title>B.n4js</title> |
| <para> |
| <programlisting language="n4js" linenumbering="unnumbered">export public class B {}</programlisting> |
| </para> |
| </formalpara> |
| <section xml:id="_step-1-convert-cursor-position-to-declared-element"> |
| <title>Step 1: Convert Cursor Position to Declared Element</title> |
| <simpara>This step is represented by the purple color in <link linkend="fig:findreference_workflow">Find reference workflow</link> diagram.</simpara> |
| <simpara>In the IDE, for the sake of convenience, we allow the user to find references of an arbitrary element at the current cursor. |
| For instance, while the cursor is currently at <literal>IdentifierRef B</literal> in the <literal>NewExpression</literal> in <literal>A.n4js</literal>, the user may want to find all references to <literal>B</literal>. |
| In those cases, we first need to find declaration element of <literal>IdentifierRef B</literal> which is <literal>TClass B</literal>. The Target URIs then contains a single URI to <literal>TClass B</literal>. |
| In diagram <link linkend="fig:findreference_workflow">Find reference workflow</link>, the classe <literal>EObjectAtOffsetHelper</literal> can convert the current cursor position into a declared element.</simpara> |
| </section> |
| <section xml:id="_step-2-convert-declared-element-to-target-uris"> |
| <title>Step 2: Convert Declared Element to Target URIs</title> |
| <simpara>This step is represented by the yellow color in <link linkend="fig:findreference_workflow">Find reference workflow</link> diagram.</simpara> |
| <simpara>The Target URIs contains the URIs whose references are to be searched. |
| The caller guarantees that <emphasis>Target URIs contain only URIs to declared elements, i.e. definitions</emphasis>. |
| For example, if we want to find references for <literal>N4ClassDeclaration B</literal> in <literal>B.n4js</literal>, the target URIs contains a URI to the AST node <literal>N4ClassDeclaration B</literal> and a URI to the TModule node <literal>TClass B</literal>. |
| Note that, in addition to the URI to the AST node <literal>N4ClassDeclaration B</literal>, the URI to the derived TModule node <literal>TClass B</literal> is also needed because <literal>N4ClassDeclaration</literal> can never be a target of a cross reference. |
| In the diagram <link linkend="fig:findreference_workflow">Find reference workflow</link> , the classes depicted in yellow color are responsible for converting declared elements to <literal>Target URIs</literal> taking care of the derived <literal>TModule</literal> nodes.</simpara> |
| </section> |
| <section xml:id="_step-3-filter-potential-resources"> |
| <title>Step 3: Filter Potential Resources</title> |
| <simpara>This step is represented by the green color in <link linkend="fig:findreference_workflow">Find reference workflow</link> diagram.</simpara> |
| <simpara>The general algorithm for finding references is to traverse the AST of each resource in the index and check each AST node if it has a cross reference to one of the URI in the <literal>Target URIs</literal>. |
| However, this is too expensive because potentially all resources in the index have to be loaded. |
| We need some way to quickly decide for a resource description if the corresponding resource may potentially contain the references before actually loading it for a more thorough search. |
| This is done using two pieces of information:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><literal>typesOrModulesToFind</literal>: the set containing the fully qualified names of the <literal>type</literal> and <literal>module</literal> of the declaration to be searched. This set is calculated in the class <literal>TargetURIKey</literal>.</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>imported names</literal>: the set exposed by <literal>ResourceDescription</literal> that contains the types needed by the underlying resource. The implementation for calculating imported names can be found in the class <literal>N4JSResourceDescription</literal>.</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>In our example, supposed that we are finding references for <literal>class B</literal>. The <literal>typesOrModulesToFind</literal> contains fully qualified names to <literal>N4ClassDeclaration B</literal> and <literal>module B</literal>, i.e. <literal>B.B</literal> and <literal>B</literal>. |
| The <literal>imported names</literal> of the resource description of <literal>A.n4js</literal> contains fully qualified names to <literal>module B</literal>, <literal>class B</literal>, i.e. <literal>B</literal> and <literal>B.B</literal>. |
| Since the set of imported names of <literal>A.n4js</literal> contains elements in <literal>typesOrModulesToFind</literal>, this resource is searched thoroughly for references.</simpara> |
| </section> |
| <section xml:id="_step-4-search-references-in-resource"> |
| <title>Step 4: Search References in Resource</title> |
| <simpara>If a resource is considered as a candidate for a more thorough search in Step 3, it is loaded. |
| Its AST is traversed and at each AST node we check if there is a cross reference to one of the Target URIs (Step 1). |
| If yes, the AST node is collected in the set of found references. |
| See class <literal>ConcreteSyntaxAwareReferenceFinder</literal> for implementation details.</simpara> |
| <simpara>The UI dependent logic may apply additional filters to drop references that are not relevant to the user, e.g. the reference from an AST element to its inferred type and vice versa (see <literal>N4JSReferenceQueryExecutor.isToBeIgnored(EReference)</literal>).</simpara> |
| </section> |
| <section xml:id="_limitations-and-possible-enhancements"> |
| <title>Limitations and Possible Enhancements</title> |
| <simpara>Other noteworthy limitations and potential enhancements of the current implementations are:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>Semantics: Only references that are available in the model as real references are reported. Even though <literal>getB()</literal> in <literal>myA.getB().getC()</literal> may return an instance of type <literal>B</literal>, there is no reference reported to B in that expression, though a reference to a member of B would be reported for <literal>getC</literal>.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Visibility constraints are not applied and thus do not reduce the search scope to allow the report of invalidly established references in a later validation.</simpara> |
| </listitem> |
| </itemizedlist> |
| </section> |
| </section> |
| </section> |
| </chapter> |
| <chapter xml:id="_compilation"> |
| <title>Compilation</title> |
| <warning> |
| <simpara>This chapter may be outdated.</simpara> |
| </warning> |
| <section xml:id="chap:compilation" role="language-n4js"> |
| <title>Introduction</title> |
| <simpara>Compilation is the process of transforming some source code written by means of a human readable text into a machine readable target file, i.e. bytecode or assembler. However, in the context of N4JS, the target output is not machine code but another high-level programming language. This kind of compiler transforming from one programming language to another is called <emphasis>transpiler</emphasis>, as a combination of <literal>transformation</literal> and <literal>compiler</literal>. The design of the transpiler takes this special setup into account.</simpara> |
| <section xml:id="sec:general_design_rationals"> |
| <title>General design rationals</title> |
| <section xml:id="sec:logging_and_error_reporting"> |
| <title>Logging and error reporting</title> |
| <simpara>The generator always expects to get a valid model as an input. So all syntactic and semantic errors (not warnings) in the N4JS code as well as regarding the project configuration, e.g. in the package.json file, should be already discovered in the validation step. So any error marker on the resource will prevent the compiler to run.</simpara> |
| <simpara>In case of other errors arising in the generator, the error handling is done as follows:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>either throw an <literal>GeneratorException</literal> or better call <literal>ExceptionHandler.handleError(message)</literal> (that then will throw this exception)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>beside the message also the file and the current line can be passed to <literal>GeneratorException</literal></simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>GeneratorException</literal> (as extending RuntimeException) will be handled by the generator caller</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>in UI: <literal>BuildInstruction</literal> will create an error log entry</simpara> |
| </listitem> |
| <listitem> |
| <simpara>headless: lets the <literal>GeneratorException</literal> fail</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| </itemizedlist> |
| </section> |
| <section xml:id="sec:progress_monitor"> |
| <title>Progress monitor</title> |
| <simpara>The compiler works on a single file and we do not expect that to take much time. Processors working on many files, such as linkers (in the JavaScript context, that is minification and concatenation), are different components.</simpara> |
| </section> |
| </section> |
| <section xml:id="sec:Xtext_Integration"> |
| <title>Xtext Integration</title> |
| <section xml:id="sec:xtext_default_behaviour"> |
| <title>Xtext default behaviour</title> |
| <simpara>The Xtext builder participant calculates the delta for every change in the project and triggers dirty state handling as well as code generation. By default the builder participant expects exactly one implementation bound to the IGenerator interface and thus only one output configuration provider belonging to this generator.</simpara> |
| </section> |
| <section xml:id="sec:n4js_requirements"> |
| <title>N4JS requirements</title> |
| <simpara>In constrast to the default Xtext behaviour in the N4JS IDE allows</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>the registration / discovery of multiple generators (including compilers and transpilers) but even no compiler at all, so that it is possible to ship the IDE also without any generators</simpara> |
| </listitem> |
| <listitem> |
| <simpara>to configure these generators separately (output paths and so on) workspace globally but also project specific (currently it is only required to enable / disable a compiler)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>to enable / disable generators, but it is allowed to enable more than one compiler at one</simpara> |
| </listitem> |
| <listitem> |
| <simpara>to start compilers headless, i.e. e.g in automated build / JUnit tests but with the possibility to configure them there as well</simpara> |
| </listitem> |
| </itemizedlist> |
| </section> |
| <section xml:id="sec:compiler_discovery_in_ui"> |
| <title>Compiler discovery in UI</title> |
| <itemizedlist> |
| <listitem> |
| <simpara>There is a singleton <literal>ICompositeGenerator</literal>. The <literal>ICompositeGenerator</literal> instance itself doesn’t contain generator logic but knows subgenerators (that implement <literal>ISubGenerator</literal>) to which it delegates the input file (so instead of registering a new builder participant (that then would have to recalculate the delta) only the generator call is extracted)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>In the UI case, the actual execution of the registered generator is done in <literal>BuildInstruction</literal> where every N4JS resource contained in the delta calculated by the builder participant is an <literal>ICompositeGenerator</literal> implementation.</simpara> |
| </listitem> |
| <listitem> |
| <simpara><xref linkend="fig:cd_GeneratorAndBuilderParticipant"/> summarizes the implementation of this compiler infrastructure in the UI case.</simpara> |
| </listitem> |
| </itemizedlist> |
| <figure xml:id="fig:cd_GeneratorAndBuilderParticipant" role="center"> |
| <title>Builder (UI) and Generator (Core)</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/13_compilation/images/cd_GeneratorAndBuilderParticipant.svg"/> |
| </imageobject> |
| <textobject><phrase>cd GeneratorAndBuilderParticipant</phrase></textobject> |
| </mediaobject> |
| </figure> |
| </section> |
| <section xml:id="sec:compiler_discovery_in_headless"> |
| <title>Compiler discovery in headless</title> |
| <itemizedlist> |
| <listitem> |
| <simpara><literal>N4JSHeadlessStandaloneSetup</literal> is used to combine <literal>N4JSRuntimeModule</literal> and <literal>N4JSHeadlessGeneratorModule</literal> so there is <literal>PropertiesFileBasedValuesProvider</literal> is bind as implementation for <literal>IPreferenceValuesProvider</literal></simpara> |
| </listitem> |
| <listitem> |
| <simpara>via <literal>N4JSHeadlessStandaloneSetup</literal> the injector is created which is used to create an instance of <literal>ICompositeGenerator</literal></simpara> |
| </listitem> |
| <listitem> |
| <simpara>in the headless mode the subgenerators of are manually registered via an extension point in the class <literal>HeadlessExtensionRegistrationHelper</literal>.</simpara> |
| </listitem> |
| </itemizedlist> |
| </section> |
| <section xml:id="sec:general_generator_implementation"> |
| <title>General generator implementation</title> |
| <example> |
| <title>Simple IGenerator</title> |
| <simpara>The following snippet shows a minimal generator example.</simpara> |
| <programlisting language="n4js" linenumbering="unnumbered">public class Generator implements IGenerator { |
| @Override public void doGenerate(Resource resource, IFileSystemAccess fsa) { |
| final String filename = computeTargetFileName(resource.getURI()); |
| |
| Script script = IterableExtensions.<Script> head( |
| Iterables.<Script> filter(resource.getContents(), Script.class)); |
| |
| fsa.generateFile(filename, doCompileToString(script)); |
| } |
| }</programlisting> |
| <simpara>Generation is triggered for each Xtext resource. <literal>IFileSystemAccess</literal> is an abstraction of where to write the generated artefacts. This enables using the generator in the IDE (with a workspace) and headless (directly operating on files). In tests you can use <literal>InMemoryFileAccess</literal>, in standalone mode you should use <literal>JavaFileSystemAccess</literal> and in Eclipse mode <literal>EclipseFileSystemAccess2</literal></simpara> |
| </example> |
| </section> |
| <section xml:id="sec:general_generator_activation"> |
| <title>General generator activation</title> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Programmatically</simpara> |
| </entry> |
| <entry> |
| <simpara>Invoke the <literal>IComposedGenerator.doGenerate</literal> with a loaded N4JS resource and a configured <literal>IFileSystemAccess</literal> implementation.</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Xtext Builder</simpara> |
| </entry> |
| <entry> |
| <simpara>This is available by default when using the bound <literal>IGenerator</literal>, it runs on every change of your N4JS resource when automatic build is enabled in the workspace.</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Context Menu</simpara> |
| </entry> |
| <entry> |
| <simpara>see <link xl:href="christiandietrich.wordpress.com/2011/10/15/xtext-calling-the-generator-from-a-context-menu/">Christian Dietrich’s Blog 2011/10/15</link></simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>MWE2 Workflow</simpara> |
| </entry> |
| <entry> |
| <simpara>via <literal>org.eclipse.xtext.generator.GeneratorComponent</literal> that is configured with the standalone setup of the N4JS language. Such an MWE2 workflow also requires the <literal>org.eclipse.xtext.mwe.Reader</literal> component to first load the N4JS resources to transform in a model slot that is then consumed by the GeneratorComponent</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| </section> |
| <section xml:id="sec:Overview_of_Input_Models"> |
| <title>Overview of the Input Models</title> |
| <simpara>The input is a simple instance of <literal>Script</literal>, which is the root model element for all N4JS files. Actually, it is the root of the AST. For the AST elements, other elements stemming from other models are accessible as well. The following models may be important for the compiler:</simpara> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>N4JS AST</simpara> |
| </entry> |
| <entry> |
| <simpara>The abstract syntax tree is an EMF model, it is defined in a single Xcore file <literal>N4JS.xcore</literal></simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Parser or Node Model</simpara> |
| </entry> |
| <entry> |
| <simpara>The parser tree, also called node model, is defined by Xtext. It contains offset information, holding whitespaces, line breaks, comments as well as other hidden tokens. It can be accessed via <literal>NodelModelUtils</literal>, that is a node can be retrieved for a given AST element and vice versa. There are three different kind of nodes: root, composite and leaf node.<?asciidoc-br?> |
| <emphasis role="strong">As of Dec 2015, the transpiler does no longer make use of the parse tree!</emphasis></simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Type Model</simpara> |
| </entry> |
| <entry> |
| <simpara>The type model is an abstract view on the N4JS AST. It is defined in a single Xcore file <literal>Types.xcore</literal>. Not all AST elements are related to type model information. This is only true for subtypes of <literal>TypeDefiningElement</literal>, with references to <literal>Type</literal> or containing a <literal>TypeRef</literal>.</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>N4 Project</simpara> |
| </entry> |
| <entry> |
| <simpara>via <literal>OutputPathHelper</literal> located in <literal>org.eclipse.n4js.generator</literal> wraps the calculation of compiled file path.</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Grammar Model</simpara> |
| </entry> |
| <entry> |
| <simpara>Grammar Model created from <literal>N4JS.xtext</literal>, the rules can be access in the Java code via <literal>N4JSGrammarAccess</literal>. The grammar elements can be retrieved from the parser model vial <literal>node.getGrammarElement()</literal>. <literal>org.eclipse.xtext.GrammarUtil</literal> also contains some useful helper methods.<?asciidoc-br?> |
| <emphasis role="strong">As of Dec 2015, the transpiler does no longer make use of the grammar model!</emphasis></simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| </section> |
| <section xml:id="sec:Core_Generator" role="language-n4js"> |
| <title>Generators</title> |
| <simpara>Generators are an abstraction above that of transpilers. N4JS transpilers are implemented as specific generators, but there might be other generators that are not transpilers (e.g. generator that produces HTML documentation from the jsdoc in the N4JS source files).</simpara> |
| <section xml:id="sec:Compiler_Components"> |
| <title>Generator Components</title> |
| <simpara><xref linkend="fig:comp_compilers"/> gives an overview over the compiler related components. Some of these components are described in detail in the following sections. |
| As of Dec 2017, the generator architecture has been refactored and simplified.</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>There is only a single <literal>ICompositeGenerator</literal> instance. Since the single instance should simply delegate to subgenerators, composite generators can no longer be registered via extension point.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Most of generator related code is moved into <literal>org.eclipse.n4js</literal> bundle. This is needed because we need to bind <literal>ICompositeGenerator</literal> to a concrete implementation in the <literal>org.eclipse.n4js</literal> bundle and the extension point for <literal>ICompositeGenerator</literal> has been removed.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>An extension point <literal>org.eclipse.n4js.generator.subgenerator</literal> is introduced in the <literal>org.eclipse.n4js</literal> bundle. This makes it possible to register a new subgenerator via extension point.</simpara> |
| </listitem> |
| </itemizedlist> |
| <figure xml:id="fig:comp_compilers" role="center"> |
| <title>Compiler Components</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/13_compilation/images/comp_compilers.svg"/> |
| </imageobject> |
| <textobject><phrase>comp compilers</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara><xref linkend="fig:od_generatorInjection"/> shows how composite generator and subgenerators interact with other components both in the UI and in the headless case.</simpara> |
| <figure xml:id="fig:od_generatorInjection" role="center"> |
| <title>Discovering generators and provide them with Guice bindings.</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/13_compilation/images/od_generatorInjection.svg"/> |
| </imageobject> |
| <textobject><phrase>od generatorInjection</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara>As we can see in the diagram above. In the UI case, <literal>N4JSBuilderParticipant</literal> creates <literal>BuildInstruction</literal> which in turn delegates the generation logics to an instance of <literal>ICompositeGenerator</literal>. The <literal>ICompositeGenerator</literal> simply delegates the generation logics to subgenerators .</simpara> |
| <simpara>In the headless mode, <literal>n4jscBase.doMain</literal> creates an instance of <literal>N4JSStandaloneSetup</literal> and obtains the injector from there. This injector is then used to create an instance of <literal>ICompositeGenerator</literal> in <literal>N4HeadlessCompiler</literal>.</simpara> |
| </section> |
| <section xml:id="sec:Generator_architecture"> |
| <title>Generator architecture</title> |
| <simpara>The compiler has to create different compilation targets, e.g., for web applications running in a browser (Chrome), or for applications running on iOS using the JavaScriptCore framework <footnote><simpara><link xl:href="https://developer.apple.com/library/mac/documentation/Carbon/Reference/WebKit_JavaScriptCore_Ref/_index.html">https://developer.apple.com/library/mac/documentation/Carbon/Reference/WebKit_JavaScriptCore_Ref/_index.html</link></simpara></footnote>. Other scenarios may include code created for debugging purposes vs. optimized code, although this may be implementable via configuration switches as well.</simpara> |
| <simpara><xref linkend="fig:cd_SubGenerators"/> shows the main generator classes, including two sub generators for EcmaScript code and EcmaScript on iOS.</simpara> |
| <figure xml:id="fig:cd_SubGenerators" role="center"> |
| <title>Generator and sub-generators</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/13_compilation/images/cd_SubGenerators.svg"/> |
| </imageobject> |
| <textobject><phrase>cd SubGenerators</phrase></textobject> |
| </mediaobject> |
| </figure> |
| </section> |
| <section xml:id="sec:Unified_Compiler_Configuration"> |
| <title>Unified Compiler Configuration</title> |
| <simpara>Since the compiler is to be used in both UI and headless (or CLI) mode, the configuration has to abstract from Eclipse <literal>IPreferenceStore</literal> concept or CLI utility classes. This is done with the combination of <literal>CompilerDescriptor</literal> and <literal>CompilerProperties</literal>, used by all <literal>ISubGenerator</literal> implementations (see <link linkend="fig:cd_SubGenerators">Fig. Sub Generators</link>).</simpara> |
| <simpara>Each compiler provides</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>a unique name (that have to match with the name of the output configuration)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>a default compiler descriptor that contains the preference values to be applied when nothing else been configured in the provided preference values</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>A <literal>CompilerDescriptor</literal> has</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>an identifier (this is the unique name of the compiler as mentioned before)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>a name (a readable name to used in Eclipse preference page)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>a description (not used yet, but maybe later also shown in the preference page)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>a flag, that indicates, if this generator should run by default</simpara> |
| </listitem> |
| <listitem> |
| <simpara>the file extension to be used for the compiled file</simpara> |
| </listitem> |
| <listitem> |
| <simpara>the <literal>OutputConfiguration</literal> object from Xtext that contains output related preferences like the output folder</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>The <literal>CompilerProperties</literal> is an enumeration that makes it easier to iterate over the preferences and getting / setting the preference values in a generic way. So this enumeration contains all configurable properties as literals.</simpara> |
| <simpara>The keys for preferences have to follow a fixed pattern as it also used internally by the builder participant when applying the configurations from the <literal>OutputConfiguration</literal>. So the key consists of</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>’outlet’</simpara> |
| </listitem> |
| <listitem> |
| <simpara>unique name of the compiler = unique name of the output configuration</simpara> |
| </listitem> |
| <listitem> |
| <simpara>the name of the property</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>Example: outlet.es5.compiledFileExtension</simpara> |
| <simpara><literal>N4JSPreferenceAccess</literal> encapsulates the access to the injected <literal>IPreferenceValuesProvider</literal>. This values provider is bound in UI to <literal>EclipsePreferencesProvider</literal> that creates an overlay over the default configuration and makes it so possible to have workspace global as well as project specific preferences and always as fall back the default values.</simpara> |
| <simpara>In headless mode the <literal>PropertiesFileBasedValuesProvider</literal> is bound as implementation of <literal>IPreferenceValuesProvider</literal>. With this implementation it is possible to load the preferences from a provided properties file.</simpara> |
| <simpara><literal>N4JSPreferenceAccess</literal> is used in <literal>AbstractSubGenerator</literal> which provided the most common used preferences as extra methods.</simpara> |
| </section> |
| </section> |
| <section xml:id="sec:Transpilers" role="language-n4js"> |
| <title>Transpilers</title> |
| <simpara>Transpilers are a special case of generators, used for transforming N4JS source code into some target code in some other, high-level programming language. In this section we describe the general transpiler infrastructure without considering any particular transpiler. Currently, there is only a single such concrete transpiler for ECMAScript target code, explained later in <xref linkend="sec:N4JS_to_EcmaScript_Transpiler"/>.</simpara> |
| <simpara>All code of the general transpiler infrastructure is found in bundle <literal>org.eclipse.n4js.transpiler</literal>.</simpara> |
| <section xml:id="sec:Phases"> |
| <title>Overview</title> |
| <simpara><xref linkend="fig:ad_PipelineOverview"/> shows an overview of the steps during transpilation of a single resource:</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>an initial conversion from the original AST to an <emphasis role="strong">intermediate model (IM)</emphasis>, called <emphasis role="strong">preparation step</emphasis>.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>one or more <emphasis role="strong">transformation</emphasis> phases, each taking as input the IM and performing a number of in-place modification on it.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>a final <emphasis role="strong">pretty printing step</emphasis> that transform the final version of the IM into the textual output, i.e. the target code.</simpara> |
| </listitem> |
| </orderedlist> |
| <figure xml:id="fig:ad_PipelineOverview" role="center"> |
| <title>Overview of the compilation pipeline</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/13_compilation/images/ad_PipelineOverview.svg"/> |
| </imageobject> |
| <textobject><phrase>ad PipelineOverview</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara>The IM is the most important data structure in the transpiler. It starts out as a 1-to-1 copy of the original AST and is then gradually transformed by the AST transformation steps into, ultimately, a representation of the output code. Only the IM undergoes updates, while the original AST remains unchanged. Nodes in the IM that are identical on N4JS source code and target code side can simply be left unchanged. Traceability links allow navigating back to an original AST node from a given IM node, but due to the gradual modification of the IM this might not be possible for all IM nodes (the tracer will return <literal>null</literal> in those cases.</simpara> |
| <simpara>Ideally, each transformation executed during the transformation step should be self-contained and coupling should be reduced to a minimum. Of course, this is not possible in all cases, in practice. Therefore, a simple mechanism is provided for statically specifying dependencies between transformations by way of Java annotations (see Java class <literal>TransformationDependency</literal> for more details). The ECMAScript transpiler, for example, has 18 individual transformations (at time of writing).</simpara> |
| </section> |
| <section xml:id="relation-between-ast-and-im"> |
| <title>Relation between AST and IM</title> |
| <simpara>The relation between the original AST and the IM is maintained by the tracer, see class <literal>Tracer</literal>, which is available via the transpiler state. The tracer allows to obtain a IM elements for a given original AST node and, conversely, original AST nodes for a given IM element (i.e. a 1:N association between original AST node and IM element).</simpara> |
| <simpara>The main purpose of this tracing information is to compute source maps for the target code.</simpara> |
| <simpara>At the beginning of the transformation step, there is a 1-to-1 correspondence between AST and IM, but over the course of the transformations this correspondence will become more and more blurred. Therefore, whenever using the tracer to get to the original AST from a given IM element , we have to consider the case that there is not original AST node defined for (because was created programmatically by an earlier transformation) OR that the original AST node is of a different kind than (because, maybe, an original N4JS class declaration was replaced by a function declaration by an earlier transformation).</simpara> |
| <simpara>Whenever a transformation changes the IM, it is responsible to update the tracer, accordingly.</simpara> |
| </section> |
| <section xml:id="implementation-overview"> |
| <title>Implementation Overview</title> |
| <simpara><link linkend="fig:transpilerClassDgr">Transpiler Class Diagram</link> shows a class diagram of the main constituents of the transpiler infrastructure.</simpara> |
| <figure xml:id="fig:transpilerClassDgr" role="center"> |
| <title>Class diagram for the transpiler infrastructure.</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/13_compilation/images/TranspilerClassDgr.svg"/> |
| </imageobject> |
| <textobject><phrase>TranspilerClassDgr</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara>The <literal>AbstractTranspiler</literal> controls the overall workflow shown earlier in . Concrete subclasses of <literal>Transformation</literal> perform the actual transformations (the preparation and pretty-printing steps are not shown in the above class diagram). Concrete transformations are created via injection within concrete sub classes of <literal>AbstractTranspiler</literal> (see class <literal>EcmaScriptTranspiler</literal> for an example). All information required during transpilation is kept in a simple data class called <literal>TranspilerState</literal>; a single instance of this class is created during the preparation step and is passed along until transpilation of the resource to transpile is completed.</simpara> |
| <simpara>Class <literal>Transformation</literal> has a super class <literal>TranspilerComponent</literal> that has two important responsibilities:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>it contains many utility methods that are easily accessible from within concrete transformations through inheritance.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>it obtains the transpiler state via injection (using the scoping feature of Google Guice, for more details see <literal>org.eclipse.n4js.utils.di.scopes.ScopeManager</literal> and <literal>TransformationScoped</literal>). This injection is done in super class <literal>TranspilerComponent</literal>, so when implementing a new transformation, the programmer does not have to deal with these details and can simply obtain the transpiler state via the inherited method <literal>TranspilerComponent#getState()</literal>.</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>Code shared across concrete transformations should be placed in sub classes of <literal>TransformationAssistant</literal>. Those assistants are similar to the helpers used elsewhere, but by sharing the <literal>TranspilerComponent</literal> super class they get all the utility methods provided by that class and they automatically get the transpiler state.</simpara> |
| <simpara>For more implementation details see the code and javadoc; a good starting point for investigating the overall workflow are classes <literal>AbstractTranspiler</literal> and <literal>Transformation</literal>.</simpara> |
| </section> |
| <section xml:id="sec:Guidelines_for_Implementing_Transformations"> |
| <title>Guidelines for Implementing Transformations</title> |
| <simpara>Some hints:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>if you need to create an entirely new transformation:</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>create new sub class of <literal>Transformation</literal> (use Xtend).</simpara> |
| </listitem> |
| <listitem> |
| <simpara>in the main class of the transpiler you are working with (probably <literal>EcmaScriptTranspiler</literal>), change method |
| <literal>#computeTransformationsToBeExecuted()</literal> to return an instance of your new transformation. The instance should be created using a Guice provider (see <literal>EcmaScriptTranspiler</literal> for an example). Note that this method also defines the order of transformations!</simpara> |
| </listitem> |
| <listitem> |
| <simpara>implement the <literal>#transform()</literal> method of your newly created transformation.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>consider adding pre and post conditions via methods <literal>#assertPreConditions()</literal> and <literal>#assertPostConditions()</literal> (throw an AssertionError if failed).</simpara> |
| </listitem> |
| <listitem> |
| <simpara>consider declaring dependencies to other transformations using the annotations defined in class <literal>TransformationDependency</literal>.</simpara> |
| </listitem> |
| </orderedlist> |
| </listitem> |
| <listitem> |
| <simpara>code shared across transformations should be placed in a new or existing sub class of <literal>TransformationAssistant</literal> and then this assistant should be injected into the transformations that require this code’s functionality.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>inside a transformation or transformation assistant:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>to modify the IM, use the utility methods inherited from <literal>TranspilerComponent</literal> (e.g. <literal>#replace()</literal>, <literal>#insertBefore()</literal>); try to avoid direct manipulation of the IM as far as possible (but sometimes it’s necessary).</simpara> |
| </listitem> |
| <listitem> |
| <simpara>to create new IM elements, use the convenience methods in <literal>TranspilerBuilderBlocks</literal>; use static import.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>to create a new symbol table entry or to obtain an existing symbol table entry for a given original target or element in the IM, use the inherited utility methods <literal>TranspilerComponent#getSymbolTableEntry*()</literal>.<?asciidoc-br?> |
| <emphasis role="strong">Never search or modify the symbol table directly!</emphasis></simpara> |
| </listitem> |
| <listitem> |
| <simpara>to access the transpiler state <footnote><simpara>but note that most utility methods obtain the transpiler state automatically; so, most of the time, you won’t need to obtain the state yourself.</simpara></footnote>, use inherited method <literal>TranspilerComponent#getState()</literal> (by convention, in Xtend you should just write <literal>state</literal> as if it were a field).</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara>for local testing, activate additional consistency checks between transformations and assertion of pre/post conditions via these boolean flags:<?asciidoc-br?> |
| <literal>AbstractTranspiler#DEBUG_PERFORM_VALIDATIONS</literal>,<?asciidoc-br?> |
| <literal>AbstractTranspiler#DEBUG_PERFORM_ASSERTIONS</literal>.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>never add one of the following replaced EMF entities to the IM:<?asciidoc-br?> |
| <literal>Script</literal>,<?asciidoc-br?> |
| <literal>IdentifierRef</literal>,<?asciidoc-br?> |
| <literal>ParameterizedTypeRef</literal>,<?asciidoc-br?> |
| <literal>ParameterizedTypeRefStructural</literal>,<?asciidoc-br?> |
| <literal>ParameterizedPropertyAccessExpression</literal>.<?asciidoc-br?> |
| Instead, use the replacement entities from <literal>IM.xcore</literal> that have the <literal>_IM</literal> suffix (e.g. <literal>IdentifierRef_IM</literal>). If you always use <literal>TranspilerBuilderBlocks</literal> as described above, you won’t run into this issue.</simpara> |
| </listitem> |
| </itemizedlist> |
| </section> |
| <section xml:id="symbol-table-in-the-im"> |
| <title>Symbol Table in the IM</title> |
| <simpara>During the preparation step, the IM is created as an exact copy of the original AST in most cases. However, to make sure the IM is self-contained and does not have any cross-references to the original AST or the original TModule and to simplify certain computations within the transformations, some AST entities are modified. For this purpose, there is a small EMF model called <literal>IM.xcore</literal>. It extends the AST model <literal>n4js.xcore</literal> and adds some elements.</simpara> |
| <simpara>Most importantly, a symbol table is created and all references of the original AST pointing to an IdentifiableElement (either in the original AST or in the TModule) are rewired to a reference to an entry in the symbol table. Those entries are of type <literal>SymbolTableEntry</literal> and occur in three special forms (there is a dedicated sub class for each case). Detailed information is provided in the javadoc of <literal>SymbolTableEntry</literal> and its sub classes and is not repeated here to avoid duplication.</simpara> |
| <simpara>The following entity replacements are done while creating the IM from the original AST and the entities without <literal>_IM</literal> must <emphasis role="strong">never</emphasis> appear in the IM:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><literal>Script</literal> <literal>Script_IM</literal></simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>IdentifierRef</literal> <literal>IdentifierRef_IM</literal></simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>ParameterizedTypeRef</literal> <literal>ParameterizedTypeRef_IM</literal></simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>ParameterizedTypeRefStructural</literal> <literal>ParameterizedTypeRefStructural_IM</literal></simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>ParameterizedPropertyAccessExpression</literal> <literal>ParameterizedPropertyAccessExpression_IM</literal></simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>For example, when having in the original AST an <literal>IdentifierRef</literal> pointing to identifiable element , then the IM will contain an <literal>IdentifierRef_IM</literal> pointing to a <literal>SymbolTableEntryOriginal</literal> with a property <literal>originalTarget</literal> pointing to .</simpara> |
| <simpara>Figures <link linkend="fig:rewire_var">Rewire Var</link>, <link linkend="fig:rewire_class">Rewire Class</link>, and <link linkend="fig:rewire_import">Rewire Import</link> show a comparison between an original AST with its original TModule and the self-contained intermediate model for a number of concrete examples.</simpara> |
| <example> |
| <title>Intermediate Models for References to Variables</title> |
| <simpara>Original AST + TModule</simpara> |
| <figure xml:id="fig:rewire_var" role="center"> |
| <title>Intermediate Model for References to Variables</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/13_compilation/images/Rewire_var_pre.png"/> |
| </imageobject> |
| <textobject><phrase>Rewire var pre</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara>Intermediate model (IM)</simpara> |
| <figure xml:id="fig:rewire_var-post" role="center"> |
| <title>Intermediate Model for References to Variables (post)</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/13_compilation/images/Rewire_var_post.png"/> |
| </imageobject> |
| <textobject><phrase>Rewire var post</phrase></textobject> |
| </mediaobject> |
| </figure> |
| </example> |
| <example> |
| <title>Intermediate Model for References to Classes</title> |
| <simpara>original AST + TModule</simpara> |
| <figure xml:id="fig:rewire_class" role="center"> |
| <title>Intermediate Model for References to Classes</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/13_compilation/images/Rewire_class_pre.png"/> |
| </imageobject> |
| <textobject><phrase>Rewire class pre</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara>Intermediate model (IM)</simpara> |
| <figure xml:id="fig:rewire_class-post" role="center"> |
| <title>Intermediate Model for References to Classes (post)</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/13_compilation/images/Rewire_class_post.png"/> |
| </imageobject> |
| <textobject><phrase>Rewire class post</phrase></textobject> |
| </mediaobject> |
| </figure> |
| </example> |
| <example> |
| <title>Intermediate Model for References to Imported Classes</title> |
| <simpara>Original AST + TModule</simpara> |
| <figure xml:id="fig:rewire_import" role="center"> |
| <title>Intermediate Model for References to Imported Classes</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/13_compilation/images/Rewire_import_pre.png"/> |
| </imageobject> |
| <textobject><phrase>Rewire import pre</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara>Intermediate model (IM)</simpara> |
| <figure xml:id="fig:rewire_import-post" role="center"> |
| <title>Intermediate Model for References to Imported Classes (post)</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/13_compilation/images/Rewire_import_post.png"/> |
| </imageobject> |
| <textobject><phrase>Rewire import post</phrase></textobject> |
| </mediaobject> |
| </figure> |
| </example> |
| </section> |
| </section> |
| <section xml:id="sec:N4JS_to_EcmaScript_Transpiler" role="language-n4js"> |
| <title>N4JS-to-EcmaScript Transpiler</title> |
| <section xml:id="sec:Overview_of_Transformations"> |
| <title>Overview of Transformations</title> |
| <simpara>The following overview will soon be outdated. Therefore:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>to find out which transformations are actually being executed and in what precise order, it is best to directly look into method:<?asciidoc-br?> |
| <literal>EcmaScriptTranspiler#computeTransformationsToBeExecuted()</literal>.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>to learn about dependencies between transformations, check the annotations of the transformation class to see if one of the dependency annotations defined in <literal>TransformationDependency</literal> are given there (though probably not all dependencies will be specified in that form).</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>The following table lists all transformation by class name in the order they are executed by the <literal>EcmaScriptTranspiler</literal>.</simpara> |
| <informaltable frame="all" rowsep="1" colsep="1"> |
| <tgroup cols="2"> |
| <colspec colname="col_1" colwidth="50*"/> |
| <colspec colname="col_2" colwidth="50*"/> |
| <tbody> |
| <row> |
| <entry align="left" valign="top"><simpara>StaticPolyfillTransformation</simpara></entry> |
| <entry align="left" valign="top"></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>MemberPatchingTransformation</simpara></entry> |
| <entry align="left" valign="top"><simpara>see <link linkend="sec:Transpiling_members">Transpiling Members</link></simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>ApiImplStubGenerationTransformation</simpara></entry> |
| <entry align="left" valign="top"></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>DestructuringTransformation</simpara></entry> |
| <entry align="left" valign="top"><simpara>turn destructuring patterns into ES5 code</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>SuperLiteralTransformation</simpara></entry> |
| <entry align="left" valign="top"><simpara>super call + super access</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>ExpressionTransformation</simpara></entry> |
| <entry align="left" valign="top"><simpara>casts, <literal>instanceof</literal>, <literal>@Promisify</literal>, …​</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>DependencyInjectionTransformation</simpara></entry> |
| <entry align="left" valign="top"></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>ClassDeclarationTransformation</simpara></entry> |
| <entry align="left" valign="top"></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>InterfaceDeclarationTransformation</simpara></entry> |
| <entry align="left" valign="top"></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>EnumDeclarationTransformation</simpara></entry> |
| <entry align="left" valign="top"></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>FunctionDeclarationTransformation</simpara></entry> |
| <entry align="left" valign="top"><simpara>turn declared function into variable declaration + function expression</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>ArrowFunction_Part1_Transformation</simpara></entry> |
| <entry align="left" valign="top"></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>BlockTransformation</simpara></entry> |
| <entry align="left" valign="top"><simpara>local arguments variable, <literal>await</literal></simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>FormalParameterTransformation</simpara></entry> |
| <entry align="left" valign="top"><simpara>variadic arguments</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>ArrowFunction_Part2_Transformation</simpara></entry> |
| <entry align="left" valign="top"></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>TrimTransformation</simpara></entry> |
| <entry align="left" valign="top"><simpara>remove TypeRefs and TypeVariables</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>SanitizeImportsTransformation</simpara></entry> |
| <entry align="left" valign="top"><simpara>remove unused imports + add missing imports</simpara></entry> |
| </row> |
| <row> |
| <entry align="left" valign="top"><simpara>ModuleWrappingTransformation</simpara></entry> |
| <entry align="left" valign="top"></entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| <simpara>The main complexity lies in the three transformations for N4JS type declarations (classes, interfaces, enums) and the related three transformations for member handling at the beginning (static polyfills, member patching, API/Impl stub generation) and the module wrapping. Up to the double horizontal line, the IM is still rather close to N4JS (e.g. still contains <literal>N4ClassDeclaration</literal>s with <literal>N4MemberDeclaration</literal>s), but after that it rapidly departs from the structure of the original AST (e.g. class declarations are broken up into a function declaration and a $<literal>makeClass</literal> call, field accessors and methods become function expressions in the properties of an object literal, fields are handled differently).</simpara> |
| </section> |
| <section xml:id="sec:Transpiling_members"> |
| <title>Transpiling members</title> |
| <simpara>When processing the members of a container type, in the standard case, the transpiler simply has to generate target code for each owned member. For inherited members no output code has to be generated, because the ordinary semantics of the Javascript prototype chain is used in the generated code.</simpara> |
| <simpara>There are, however, special cases when output code has to be generated for a non-owned or non-existant member of a container type:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>partial shadowing caused by lone field accessors, [sec:Transpiling_members__Partial_shadowing_of_getter_setter_pairs]<?asciidoc-br?> |
| ( <emphasis role="strong">delegation</emphasis>)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>consumption of members of an interface within an implementing class, [sec:Transpiling_members__Consuming_or_inheriting_members_of_an_interface]<?asciidoc-br?> |
| ( <emphasis role="strong">delegation</emphasis>, for data fields: <emphasis role="strong">copying</emphasis>)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>inheritance of members of an interface within an extending interface, [sec:Transpiling_members__Consuming_or_inheriting_members_of_an_interface]<?asciidoc-br?> |
| ( <emphasis role="strong">delegation</emphasis>, for data fields: <emphasis role="strong">copying</emphasis>)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>mixing in members into a container type via static polyfill, [sec:Transpiling_members__Static_polyfill]<?asciidoc-br?> |
| ( <emphasis role="strong">copying</emphasis>)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>adding an API / implementation stub, [sec:Transpiling_members__API_implementation_stubs]<?asciidoc-br?> |
| ( <emphasis role="strong">creation</emphasis>)</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>The above overview also states what technique is used in each special case of member handling: <emphasis role="strong">delegation</emphasis>, <emphasis role="strong">copying</emphasis> or <emphasis role="strong">creation</emphasis>. Delegation is the most tricky one and means that not a new function is generated in the output code for the special member, but the existing member function of an existing member is obtained from somewhere in the prototype chain and used directly as the member function of the special member. <emphasis role="strong">Copying</emphasis> means that an existing member is copied to another location where the special handling is required as if it were defined in that place. Lastly, <emphasis role="strong">creation</emphasis> means that an entirely new member is created for which no existing member serves as a template and this member gets a body with some <literal>default</literal> behavior. These three techniques of special member handling are explained in more detail in <xref linkend="sec:Transpiling_members__Delegating_members"/>.</simpara> |
| <section xml:id="sec:Transpiling_members__Delegating_members"> |
| <title>Techniques for special member handling</title> |
| <simpara>If output code has to be generated for a non-owned member of a classifier we distinguish the following two cases:</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>either some other member owned by classifier serves as a template for ,</simpara> |
| </listitem> |
| <listitem> |
| <simpara>or no such template exists.</simpara> |
| </listitem> |
| </orderedlist> |
| <simpara>In the first case, we can either <emphasis role="strong">copy</emphasis> in the sense that we will generate output code for within the output code for as if had been defined in . Or we can use <emphasis role="strong">delegation</emphasis>, i.e. generate output code for that reuses the existing member function of in . In case no template exists, we always have to <emphasis role="strong">create</emphasis> from scratch, i.e. generate output code as if had been defined with some behavior pre-defined by the N4JS language (this applies only to API / implementation stubs where this pre-defined behaviour is to throw an <literal>unimplemented member</literal> error).</simpara> |
| <simpara>Creation and copying is straightforward; for more details on member delegation see class <literal>DelegationAssistant</literal> and entity <literal>DelegatingMember</literal> in <literal>IM.xcore</literal>. The basic approach is to allow one transformation to create a <literal>DelegatingMember</literal> and insert it into the IM and let the transformations for class and interface declarations turn this member into low-level Javascript constructs that perform the actual delegation.</simpara> |
| </section> |
| <section xml:id="sec:Transpiling_members__Partial_shadowing_of_getter_setter_pairs"> |
| <title>Partial shadowing</title> |
| <simpara>In Javascript, if an object has a setter of name , then a read access <literal>obj.prop</literal> will return undefined, even if the prototype of has a getter or field of name . Conversely, if has a getter , then a write access <literal>obj.prop = 42</literal> will produce a runtime error, even if the prototype of has a setter or field .</simpara> |
| <screen>var proto = { |
| get prop1() { return "this won't show up" }, |
| set prop2(value) { console.log("this won't be reached") } |
| } |
| var obj = { |
| set prop1(value) {}, |
| get prop2() {} |
| } |
| obj.__proto__ = proto; |
| |
| console.log(typeof obj.prop1); // will print "undefined" |
| obj.prop2 = 42; // error: "setting a property that has only a getter"</screen> |
| <simpara>Note, in plain N4JS a validation enforces a redefinition of accessors or overriding of a field always by getter/setter pairs. However, in special situations of incomplete API implementations stubs for missing accessors are created in order to provide meaningful test-reporting. This leads to situations where on the implementation side a single getter or or a single setter is defined in a subclass - unaware of possibly injected stubs in superclasses. The aforementioned validation can not enforce the user to author an accessor pair. To keep a meaningful test-response the transpiler treats this situation as follows:</simpara> |
| </section> |
| <section xml:id="sec:Transpiling_members__Consuming_or_inheriting_members_of_an_interface"> |
| <title>Consuming or inheriting members of an interface</title> |
| <simpara>When an N4JS class consumes the member of an interface implemented by , then this cannot be handled by the native prototype chain mechanism of Javascript. Instead, the transpiler has to generate a member of corresponding type that delegates to the consumed member. In case of data fields, such a delegation is not possible and thus the transpiler generates output code for the consumed data field as if the field had been defined in .</simpara> |
| <simpara>Of particular importance in this context is the diamond problem when consuming members from an interface. For example, if interface defined method with a default implementation, interface extends and overrides with a different implementation, class implements and class extending implements , then will not consume because it has already inherited from its super class (which in turn has consumed it from ). So, in the default implementation of given in will be active, not that given in .</simpara> |
| </section> |
| <section xml:id="sec:Transpiling_members__Static_polyfill"> |
| <title>Static polyfill</title> |
| <simpara>See class <literal>StaticPolyfillTransformation</literal> for details.</simpara> |
| </section> |
| <section xml:id="sec:Transpiling_members__API_implementation_stubs"> |
| <title>API / implementation stubs</title> |
| <simpara>See <link linkend="sec:Support_for_incomplete_API_implementation_testing_in_the_N4JS_to_EcmaScript_5_Transpiler">Support for incomplete API implementation testing</link>.</simpara> |
| </section> |
| </section> |
| <section xml:id="sec:Support_for_incomplete_API_implementation_testing_in_the_N4JS_to_EcmaScript_5_Transpiler"> |
| <title>Support for incomplete API implementation testing</title> |
| <simpara>As part of the introduction of API projects with executable test cases the need to verify the state of implementations came into focus. No formal dependency is allowed between an API project and its dedicated implementation projects, hence an inconsistency can not directly be detected. However at runtime (c.f. <link linkend="_execution">Execution</link>) the API is always replaced by an appropriate implementation.</simpara> |
| <simpara>In cases where such an implementation is incomplete this would result in failures due to missing concepts, e.g. calls to methods that are not in place or usage of fields which are not defined. In order to support the author of an implementation the IDE provides a mean to compare the current state of implementation to the developer in a tabular way (c.f. [<link linkend="N4JSSpec">N4JSSpec</link>]).</simpara> |
| <simpara>The key idea for automated test-support is to incorporate those comparison-results into the transpiled output in way, that a test-framework can easily distinguish wrong implementations from incomplete implementations. Of course this is not always possible, but the majority of cases can be handled.</simpara> |
| <simpara>As there is only one transpilation mode the resulting modifications are always part of the generated code.</simpara> |
| <simpara>In order to distinguish the different project-types we distinguish between different project types:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>an API-project (API)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>an API-implementation project (Impl)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>a client project (Client)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>a test project (Test)</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>The API-project defines the requirements to it’s implementors in form of definition-files (n4jsd). The API is defined together with an test-project which validates the implementation. Client code is written with a formal dependency to the API and uses the elements declared therein. In that sense an API-testing project is just a normal client project with a test-nature.</simpara> |
| <simpara>Additional code to support API implementation testing is only inserted into the Impl-projects. API, Client and Test are not affected.</simpara> |
| <simpara>One major goal in transpiling Impl projects is to provide the ability to load all modules used in Client/Test projects in non-disruptive way. Even if the implementation is missing elements the runtime should still be able to successfully load the module. Errors should only be signalled when the client code actually uses the missing concepts.</simpara> |
| <section xml:id="sec:Modifications_in_Impl_projects"> |
| <title>Modifications in Impl projects</title> |
| <simpara>The generator is module driven. In case of missing modules nothing will be done but the runtime will detect this and act accordingly.</simpara> |
| <simpara>In general only missing elements will be inserted:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>Missing class - a stub will be generated</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Missing function - a stub will be generated</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Missing enumeration - a stub will be generated</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Missing interface - a stub will be generated</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>Missing members of classes are inserted as stubs. Missing fields will be replaced by getter/setter-pairs throwing an error upon read and write access.</simpara> |
| <simpara>A more sophisticated approach needs to be taken for interfaces with default implementations (marked with @ProvidesDefaultImplementation or @ProvidesInitializer).</simpara> |
| <simpara>Currently missing field initialisers in interfaces are not detected for two reasons: Field-initialising is carried out on loading. Throwing an error in the initialiser will prevent the module from being loaded. Installing a getter/setter pair on the Impl-interface is not an option since the inheritance chain used in client project has no knowledge about this and therefore these accessors cannot be reached from client code.</simpara> |
| <simpara>Missing default implementations will be inserted as stubs. For normal class compilation the inheritance chain needs to be scanned. In case of an missing default implementation in an implemented interface a forwarding call to the stub needs to be inserted on the class.</simpara> |
| </section> |
| <section xml:id="sec:Implementation_of_stub_generation"> |
| <title>Implementation of stub-generation</title> |
| <simpara>The implementation is mainly organised in <literal>ApiImplStubGenerationTransformation</literal>, which makes use of <literal>MissingApiMembersForTranspiler</literal> and <literal>ScriptApiTracker</literal>.</simpara> |
| <simpara>When a Module is transpiled the type of the project is checked. Only if the project is an implementation project the comparison between the current module and it’s API module is computed and attached as an Adapter to the <literal>Script</literal> during the life-cycle of the <literal>ScriptTranspilerPolicy</literal>. The <literal>ProjectComparisonAdapter</literal> serves as a shared information pool among the different transpiler-policies and caches different compare-results. After transpilation of the script the adapter will be removed.</simpara> |
| <simpara>In order to reuse all existing code as far as possible, missing elements are modelled as subclasses of the requested element but with no AST-information. These subclasses are member-classes of the <literal>ScriptApiTracker</literal> class. All class-names are prefixed with <literal>VirtualApi</literal> and hold a reference to the type-information computed by the module-comparison.</simpara> |
| <simpara>It is important to not access the AST-elements of type-information obtained from the project-comparison, since this would trigger the AST-loading and invalidate (proxifying) existing <literal>EObjects</literal>.</simpara> |
| </section> |
| </section> |
| </section> |
| <section xml:id="sec:n4jsc_Headless_Compiler_Interface" role="language-n4js"> |
| <title>n4jsc Headless Compiler Interface</title> |
| <simpara>The headless compiler interface consists of a runnable class capable of reading command line options packaged together with all required dependencies into one executable <literal>.jar</literal> archive.</simpara> |
| <simpara>The sources of the command line interface are located in in the <literal>tools/</literal> subfolder of the main git repository. They comprise of the package <literal>org.eclipse.n4js.hlc</literal> and corresponding test package <literal>org.eclipse.n4js.hlc.tests</literal>. (c.f. <link linkend="sec:tools">Tools</link>).</simpara> |
| <simpara>A description of how to use the headless compiler can be found in the N4IDE document.</simpara> |
| <section xml:id="sec:building_the_headless_compiler"> |
| <title>building the headless compiler</title> |
| <simpara>The maven-modules related to the headless compiler build are organised in the <literal>tools</literal> folder in the project-root. In order to build the headless compiler as part of the common build-chain, the maven-profile <literal>buildTools</literal> needs to be activated (off by default), e.g. <literal>mvn -PbuildTools</literal> .</simpara> |
| <simpara>To build the headless compiler separately, the project-pom can be set to <literal>tools/pom.xml</literal>, however then the tycho-generated artefacts must be accessible by this build. This can be achieved in three different ways:</simpara> |
| <orderedlist numeration="loweralpha"> |
| <listitem> |
| <simpara>the build takes place on a workspace formerly building the n4js-project without cleaning it (interesting for local experiments),</simpara> |
| </listitem> |
| <listitem> |
| <simpara>a former n4js-build installed the artefacts into the currently configured local-.m2-repository of maven or</simpara> |
| </listitem> |
| <listitem> |
| <simpara>a former n4js-build deployed the artefacts to the configured nexus-server.</simpara> |
| </listitem> |
| </orderedlist> |
| <simpara>Note however, on our build-server option a) is not feasible, option b) requires you to setup a build-chain and ensuring the same build-node to be building on |
| and c) is difficult up to nearly impossible without a proper versioning-scheme.</simpara> |
| <simpara>Parameters for the headless-compiler-build are defined in the parent-pom located at <literal>releng/org.eclipse.n4js.parent/pom.xml</literal> with properties prefixed by <literal>hlc.</literal>.</simpara> |
| </section> |
| </section> |
| </chapter> |
| <chapter xml:id="_execution"> |
| <title>Execution</title> |
| <warning> |
| <simpara>This chapter may be outdated.</simpara> |
| </warning> |
| <simpara>There are many different use cases for executing N4JS code:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>running project locally</simpara> |
| </listitem> |
| <listitem> |
| <simpara>running tests in CI</simpara> |
| </listitem> |
| <listitem> |
| <simpara>running application in the client</simpara> |
| </listitem> |
| <listitem> |
| <simpara>running processor on the server</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>All those use cases may differ in their details, but can be divided into general phases:</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>execution environment preparation</simpara> |
| </listitem> |
| <listitem> |
| <simpara>bootstrapping</simpara> |
| </listitem> |
| <listitem> |
| <simpara>call to given n4js entry point</simpara> |
| </listitem> |
| <listitem> |
| <simpara>shutdown (optional)</simpara> |
| </listitem> |
| </orderedlist> |
| <simpara>When N4JS execution is triggered, proper <emphasis>Runner</emphasis> (see <link linkend="sec:Runners-introduction">Runners</link>) is selected. In some cases it is done automatically, in others user needs to make a choice. Runner is responsible for perform all required preparations, according to <link linkend="sec:N4JS_Project_Execution_And_Linking_Model">N4JS Project Execution And Linking Model</link>. Then JS execution environment (e.g. NodeJS, IOJS, Chrome, JavaScriptCore) performs bootstrapping according to <xref linkend="sec:N4JS_Execution_And_Linking_File"/>. As last step of bootstrap phase defend n4js entry point will be called which starts proper n4js execution phase. In some cases there may be shutdown phase, but that is highly dependent on use case and proceeding execution phases.</simpara> |
| <section xml:id="sec:N4JS_Project_Execution_And_Linking_Model" role="language-n4js"> |
| <title>N4JS Project Execution And Linking Model</title> |
| <simpara>N4JS project is compiled to JavaScript language, that in turn can be executed in some JS execution environment, Those environments (e.g. NodeJS, IOJS, Chrome, JavaScriptCore) will differ between each other in terms of JS APIs they expose and way JS code has to be provided to them, or the way it is triggered. We introduced systematic way of describing those features in terms of N4JS projects (see Components and Projects <link linkend="sec:N4_Components_and_IDE_Support">Components and IDE Support</link>). N4JS project will be of different <emphasis>PojectType</emphasis> that determines project purpose (see Package.json section <xref linkend="sec:Package_json"/>. When we want to execute some N4JS project, we can divide its dependency graph into 4 general areas</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>User Space, e.g. user code</simpara> |
| </listitem> |
| <listitem> |
| <simpara>System Space, e.g N4 Platform APIs</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Runtime Space, e.g. EcmaScript APIs</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Environment Space, e.g. execution environment APIs</simpara> |
| </listitem> |
| </orderedlist> |
| <simpara>Example of that kind of graph can bee seen on <xref linkend="fig:od_sampleProjectDependencyGraph"/></simpara> |
| <figure xml:id="fig:od_sampleProjectDependencyGraph" role="center"> |
| <title>Sample Project Dependency Graph</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/14_execution/images/od_sampleProjectDependencyGraph.svg"/> |
| </imageobject> |
| <textobject><phrase>od sampleProjectDependencyGraph</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara>All dependencies are compile time dependency (as they are checked by the compiler), but <emphasis>tend</emphasis> to weaken, the lower in the dependency graph we are. <emphasis>User Space</emphasis> objects will have strong load time and run time dependency to each other and to the <emphasis>System Space</emphasis>. <emphasis>System Space</emphasis> have strong load time and run time dependency to each other and, only runtime dependency to <emphasis>Runtime Space</emphasis>. <emphasis>Runtime Space</emphasis> objects should not have any load time dependencies between each other. In some cases they may have weak runtime dependency to each other. In many cases those components are just api definitions that describe execution environment native apis, but may contain polyfills code. <emphasis>Environment Space</emphasis> has no dependency to other components, except the fact different <emphasis>RuntimeEnvironemnt</emphasis>s can extend each other (see <xref linkend="sec:N4_Components_and_IDE_Support"/>).</simpara> |
| <simpara>Runner must configure JS execution environment in the way that all above areas of the dependency graph must be either</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>provided by execution environment itself (runtime libraries APIs - <emphasis>n4jsd</emphasis> files)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>loaded by defined runtime environment (self initialisation code)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>available to load by environment explicitly (runtime libraries polyfills, system libraries)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>available to load by other implicitly (system libraries, user libraries and projects)</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>Testers, the same way as runners, must be able to execute n4js code. Main difference is that dependency graph for test case will be usually slightly bigger (dependencies to test libraries), and code that has to be triggered shifts a bit from given project to test library used in test code of tested project. Extending previously used example with test elements is shown in figure <xref linkend="fig:od_sampleTestProjectDependencyGraph2"/>.</simpara> |
| <figure xml:id="fig:od_sampleTestProjectDependencyGraph2" role="center"> |
| <title>Sample Test Project Dependency Graph</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/14_execution/images/od_sampleTestProjectDependencyGraph.svg"/> |
| </imageobject> |
| <textobject><phrase>od sampleTestProjectDependencyGraph</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <figure xml:id="fig:runners-testers" role="center"> |
| <title>Runners and Testers</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/14_execution/images/runners-testers.svg"/> |
| </imageobject> |
| <textobject><phrase>runners testers</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <section xml:id="subsec:N4JS_Execution_With_NodeJS" role="language-n4js"> |
| <title>N4JS Execution With NodeJS</title> |
| <simpara>This example shows in-depth details of N4JS code execution with NodeJS runner.</simpara> |
| <simpara>In the workspace we have <literal>Client</literal> with <literal>foo.n4js</literal> that imports <literal>bar.n4js</literal> from <literal>UserLib</literal> that is also in the workspace. |
| Those N4JS files use some ES5 APIs , e.g. <literal>Math.random()</literal> and <literal>setTimeout()</literal>. Those APIs are <literal>Global</literal> so there is |
| no impicit import, still they make user projects depend on runtime library <literal>n4js-runtime-es2015</literal>. |
| Assuming user selects <literal>foo.n4js</literal> file for execution, the NodeRunner in the IDE will:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>create working directory in temp folder, e.g. <literal>/var/temp/N4JSNodeRun123/</literal></simpara> |
| </listitem> |
| <listitem> |
| <simpara>create <literal>node_modules</literal> folder inside working directory, to which projects will be linked</simpara> |
| </listitem> |
| <listitem> |
| <simpara>generate script, e.g. <literal>n4jsELF.js</literal> that will be responsible for booting execution (see <xref linkend="sec:N4JS_Execution_And_Linking_File"/>)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>runner will put <literal>/var/temp/N4JSNodeRun123/node_modules</literal> into <literal>NODE_PATH</literal></simpara> |
| </listitem> |
| <listitem> |
| <simpara>execute <literal>/var/temp/N4JSNodeRun123/n4jsELF.js</literal> with NodeJS</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>For example with NodeJS environment if all projects from dependency graph are accessible in local file system, their |
| paths would need to be put in NodeJS <emphasis>NODE_PATH</emphasis> environment variable. In addition to configuring execution environment |
| <emphasis>Runner</emphasis> generates N4JS Elf file that is used by environment to bootstrap n4js execution |
| (see <xref linkend="sec:N4JS_Execution_And_Linking_File"/>). |
| [[fig:od_sampleNodeProjectExecution]</simpara> |
| <figure role="center"> |
| <title>Sample NodeJS Project Execution</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/14_execution/images/od_sampleNodeProjectExecution.svg"/> |
| </imageobject> |
| <textobject><phrase>od sampleNodeProjectExecution</phrase></textobject> |
| </mediaobject> |
| </figure> |
| </section> |
| </section> |
| <section xml:id="sec:N4JS_Execution_And_Linking_File" role="language-n4js"> |
| <title>N4JS Execution And Linking File</title> |
| <simpara>JS execution environment not only needs to know from where it needs to obtain code to execute, but also <emphasis>what is the entry point to the code that is supposed to be executed</emphasis> and <emphasis>what code needs to be loaded before entry point is called</emphasis>.</simpara> |
| <simpara>All this information is generated by the runner based on the executed project dependency graph. The way this information |
| is presented depends on concrete JS execution environment used, and on its configuration (e.g. user provided options, or |
| configuration derived in other ways). But in either case file is generated with that information. Figure <xref linkend="fig:n4js_elf"/> |
| shows examples how this information would look like for both testers and runners for both NodeJS or Chrome.</simpara> |
| <figure xml:id="fig:n4js_elf" role="center"> |
| <title>N4JS ELF examples</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/14_execution/images/n4js_elf.svg"/> |
| </imageobject> |
| <textobject><phrase>n4js elf</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara>First segment of the n4js elf is responsible for loading <emphasis>RuntimeEnvironment</emphasis> bootstrap code. Since <emphasis>RuntimeEnvironment</emphasis>s can extend each other, generated information would follow those dependencies. It is possible that <emphasis>RuntimeEnvironment</emphasis>s need to do some special work in regards of of provided <emphasis>RuntimeLibraries</emphasis>s, e.g. initiate initiate polyfills. That code can be either directly in <emphasis>RuntimeEnvironment</emphasis> init code, or its init code can call modules from provided runtime libraries.</simpara> |
| <simpara>Second segment of the n4js elf is responsible for loading <emphasis>RuntimeEnvironment</emphasis> exec module. This is special module defined in package.json of the environment, that is used to call into user projects entry point directly, or (as in test case) call into runners of the test library.</simpara> |
| <simpara>Last segment of the n4js elf is responsible for passing run/test data (generated by IDE/CLI) into initialised previously exec module.</simpara> |
| <simpara>While first two segments are resolved from project dependencies and can be covered by generic approach on IDE/CLI side, last segment requires strong relation between given runner/tester and <emphasis>RuntimeEnvironment</emphasis> / <emphasis>TestEnvironment</emphasis>. While some generic approaches can be used, for the moment we don’t specify concrete convention there.</simpara> |
| <section xml:id="subsec:NodeJS_Specific_ELF" role="language-n4js"> |
| <title>NodeJS Specific ELF</title> |
| <simpara>Concrete environments may need specific setup that is not common for other environemtns. For example for NodeJS runner |
| needs to configure the node lookup paths for the module resolution. This is achieved by creating at runtime symlinks |
| from <literal>node_modules</literal> pointing to concrete dependencies required during execution.</simpara> |
| </section> |
| </section> |
| <section xml:id="sec:Runners-execution" role="language-n4js"> |
| <title>Runners</title> |
| <simpara>It is specified above, that <emphasis>Runner</emphasis> prepares concrete JS execution environment for executing given code and triggers execution process. What is not clear so far is how appropriate runner is selected for given project. In <link linkend="_n4components">N4 Components</link> it was specified that N4JS projects do not depend directly on specific runners or JS execution environments. Instead, N4JS tooling should be able to select appropriate runner based on given project transitive dependencies. In this section we specify overall design of runners for both N4JS IDE and CLI tooling and how runners are selected for projects.</simpara> |
| <simpara>INFO: In general any n4js code execution is governed by <emphasis>runners</emphasis> and <emphasis>testers</emphasis> depending on the use case. |
| In this chapter <emphasis>runners</emphasis> are described in detail. |
| Information from this chapter applies to <emphasis>Testers</emphasis>, unless stated otherwise in chapter dedicated to testing N4JS code (<link linkend="_tests">Tests</link>), were we specify |
| testing specific use cases.</simpara> |
| <section xml:id="subsec:N4_Runtime_Environments_Convention"> |
| <title>N4 Runtime Environments Convention</title> |
| <simpara>Dependency between <emphasis>Runner</emphasis> and <emphasis>Runtime Environment</emphasis> crosses technical boundary between N4JS Projects (N4JS code) and N4JS tooling (IDE and CLI tools implemented with e.g. Java). We introduce convention to implement this dependency, yet letting N4JS projects and N4JS tools internals to be relatively independent.</simpara> |
| <figure xml:id="fig:cd_EnvironmentConvention" role="center"> |
| <title>Runtime Environments Convention</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/14_execution/images/cd_EnvironmentConvention.svg"/> |
| </imageobject> |
| <textobject><phrase>cd EnvironmentConvention</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara><xref linkend="fig:cd_EnvironmentConvention"/> convention that is used to communicate run time configuration of the N4JS projects (grey colour) and N4JSIDE (pink colour). JS projects declare dependencies on provided list of <emphasis>Runtime Libraries</emphasis>. Each combination of those corresponds to one predefined <emphasis>Runtime Environment</emphasis> N4JS component. On N4JSIDE side there is separate list of <emphasis>runtime environment</emphasis>s maintained. Both lists correspond one to one to each other.</simpara> |
| </section> |
| <section xml:id="subsec:Passing_Information_from_IDE_to_Execution_Code_in_Runtime_Environment" role="language-n4js"> |
| <title>Passing Information from IDE to Execution Code in Runtime Environment</title> |
| <simpara>When launching an N4JS file, the IDE will compute some information on the containing N4JS project and its direct and indirect dependencies as well as the runtime environment in use. This information will be passed on to the execution module defined in the runtime environment, i.e. the code specified via property in the runtime environment’s package.json file. The information will be passed via a global variable <literal>$executionData</literal>. The value will be a Javascript object with the following properties:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><literal>userSelection</literal>: the module the user had selected when initiating the launch. This will usually be the <literal>module to run</literal> , but in case of testing it will be the project, folder, or file the user had selected.</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>projectNameMapping</literal>: an object in which every key is the name of an API project among the direct or indirect dependencies of the project being run, and every value is the name of the corresponding implementation project being used. When running N4JS projects that do not make use of the API / implementation project technique, then this property will either hold an empty object or be undefined.</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>testTree</literal> (only when running tests): the test tree as defined in <link linkend="_tests">Tests</link> containing information on the tests to be run, i.e. test classes, test methods, etc. The test tree will be encoded as JSON, so the value of this property will be of type string and should be passed to <literal>JSON.parse()</literal>.</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>All calculations described above are based on the workspace available. This includes library manager functionality, see <link xl:href="../20_externalLibraries/externalLibraries.xml#:External_Library_Workspace">External Library Workspace</link>. |
| In specific setups, where workspace is not available runners provide helper utility <literal>org.eclipse.n4js.runner.RunnerFileBasedShippedCodeConfigurationHelper.configureFromFileSystem()</literal> that allows to configure given <literal>RunConfiguration</literal> using |
| plain file system to the external libraries. Note that in order to do this in a way that allows to re-use all computation logic based on <link xl:href="../12_n4components/n4components.xml#:N4MFContainerManagement">N4Containers</link>, |
| runners infrastructure provides its own subclasses of the few component types. Those specialized types are used only in scope of <literal>RunnerFileBasedShippedCodeConfigurationHelper</literal> and are not exposed to the rest of the system.</simpara> |
| <simpara>Specific runners, e.g. the NodeJS or Chrome runner, may choose to provide more information via the execution data object.</simpara> |
| </section> |
| <section xml:id="subsec:Runners_Design"> |
| <title>Runners Design</title> |
| <simpara>As specified in section before N4JS projects will need to be executed on various JS execution environments, for which dedicated runners will be needed. While they will differ how they interact with concrete JS environment, they will have common parts when it comes to interaction with N4JS IDE or CLI. Those parts are provided in form of abstract <emphasis>IDERunner</emphasis> <emphasis>CLIRunner</emphasis> and <emphasis>Runner</emphasis> <emphasis>components</emphasis> (or bundles) that specific runners should use to interact with N4JS IDE or CLI.</simpara> |
| <simpara>Runner by design consists of three parts:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><emphasis>core</emphasis> part (green colour) - contains most logic, resources (e.g. JS execution environment binary)</simpara> |
| </listitem> |
| <listitem> |
| <simpara><emphasis>IDE</emphasis> part (blue colour) - responsible for working with N4JS IDE (enabling runner in ui, providing views)</simpara> |
| </listitem> |
| <listitem> |
| <simpara><emphasis>CLI</emphasis> part (yellow colour) - responsible for working with N4JS CLI (get command line parameters, provide console output)</simpara> |
| </listitem> |
| </itemizedlist> |
| <figure xml:id="cd_RunnersIdeCli" role="center"> |
| <title>Runner for N4JS IDE and CLI tooling</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/14_execution/images/cd_RunnersIdeCli.svg"/> |
| </imageobject> |
| <textobject><phrase>cd RunnersIdeCli</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara>Specific runner is connected to running N4JS IDE or CLI via extension points. This is done either by using them directly, or by using types exposed by abstract runner component.</simpara> |
| </section> |
| </section> |
| <section xml:id="sec:Legacy_Execution_Engine" role="language-n4js"> |
| <title>Legacy Execution Engine</title> |
| <simpara>Compilation of N4JS may target many platforms. For the moment it is hard to discuss what they will be exactly or if N4JSIDE will provide some integration or hooks to those platforms. On the other hand we want to have some execution environment for internal use to validate behaviour of compiled code. Since we know that V8 based platforms (e.g. Chrome, NodeJS) will be in our target platforms set we want to be able to execute compiled code on similar environment. As standalone V8 integration is quite challenging, we have decided to integrate in N4JSIDE NodeJS as execution environment. This is considered internal feature used for testing compilation of N4JS and N4JSIDE.</simpara> |
| </section> |
| <section xml:id="sec:Design" role="language-n4js"> |
| <title>Design</title> |
| <simpara>We provide NodeJS binaries for various OSes. Direct access to binaries is not exposed. Selection and institutionalization of the binary is done internally and is not configurable. Instead bundle containing binaries provides classes required to run code (in form of <literal>String</literal> or <literal>File</literal> with the code). Clients That want to do this may either use provided <literal>Engine</literal> class or can implement their own engine based on provided infrastructure. Main class that used in engine implementation is <literal>EngineCommandBuilder</literal>. This class is responsible for building proper command line commands that engine implementation must execute to run code with NodeJS.</simpara> |
| <simpara><xref linkend="fig:cd_executionengine"/> shows the most important classes of the NodeJS integration.</simpara> |
| <figure xml:id="fig:cd_executionengine" role="center"> |
| <title>NodeJS execution integration</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/14_execution/images/cd_executionengine.svg"/> |
| </imageobject> |
| <textobject><phrase>cd executionengine</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <section xml:id="sec:Usage_Outside_N4JSIDE"> |
| <title>Usage Outside N4JSIDE</title> |
| <simpara>In this use case we use provdied <literal>Engine</literal> class that allows to execute js code in form of <literal>String</literal> or <literal>File</literal> with the code. In return user receives <literal>EngineOutput</literal> object with two lists of strings containing standard output and error output of the node process, that were captured during execution.</simpara> |
| <simpara>In this usage scenario execution api assumes valid JS code. User needs to ensure compilation of code prior to execution, if needed.</simpara> |
| <simpara>That functionality is used in internal jUnit tests and in xpect tests of the compiler.</simpara> |
| <section xml:id="sec:Use_Node_with_Maven"> |
| <title>Use Node with Maven</title> |
| <simpara>Note on maven usage. For maven based builds we need to ensure that binary resources are available and are unpacked. To do this in pom of the project that will be calling engine we must include following listing:</simpara> |
| <programlisting language="xml" linenumbering="unnumbered"> <plugin> |
| <groupId>org.apache.maven.plugins</groupId> |
| <artifactId>maven-dependency-plugin</artifactId> |
| <executions> |
| <execution> |
| <id>unpack</id> |
| <phase>process-test-classes</phase> |
| <goals> |
| <goal>unpack</goal> |
| </goals> |
| <configuration> |
| <artifactItems> |
| <artifactItem> |
| <groupId>org.eclipse.n4js</groupId> |
| <artifactId>org.eclipse.n4js.js.engine</artifactId> |
| <version>0.0.1-SNAPSHOT</version> |
| <overWrite>true</overWrite> |
| <outputDirectory> |
| ${project.build.directory}/classes |
| </outputDirectory> |
| </artifactItem> |
| </artifactItems> |
| </configuration> |
| </execution> |
| </executions> |
| </plugin></programlisting> |
| </section> |
| </section> |
| <section xml:id="sec:Usage_Inside_N4JSIDE"> |
| <title>Usage Inside N4JSIDE</title> |
| <simpara>In Eclipse platfrom based environment we use custom implementation of the engine, <literal>PlatformEngine</literal>. This implementation unlike default <literal>Platform</literal> is non blocking implementation that forwards output to platform console and allows platform to control lifecycle of the running engine. Additionally this version uses its own implementation of the <literal>ResourceUrlResolver</literal>. It is required to properly resolve urls that point inside platform bundles.</simpara> |
| <simpara>This scenario we assume that user will run N4JS files or JS files. We have created proper UI hooks that allow user to do this either from editor or as selection menu. Based on name of the file that user commands to execute we find proper compiled file (compiled with our ES5 compiler - it is not configurable). When found we execute this file in our execution engine. If it is not found, execution engine will write appropriate error to N4JSIDE console.</simpara> |
| </section> |
| </section> |
| <section xml:id="sec:Runtime_Injection" role="langauge-n4js"> |
| <title>Runtime Injection</title> |
| <simpara>There is need to inject into runtime environment some special code, for example when compiling N4JS to ES5 (see N4JavaScriptSpecification,chapter N4 JS Compilation). To achieve this wee need to inject desired code when calling engine to run desired compiled code. Injection mechanism depends a lot on way we run engine. In this section injection of runtime is discussed based on NodeJs that is used as runtime environment.</simpara> |
| <section xml:id="sec:Running_String_Code"> |
| <title>Running String Code</title> |
| <simpara>We allow code execution where code is provided in form of <literal>String</literal>. In this case we are calling nodejs with parameters . To enrich execution environment in this case we are appending special runtime code at the end of file. It is important to append it at the end, to avoid changing line numbers of original code and decrease other potential side effects. So actual invocation of nodejs looks like</simpara> |
| <simpara>This mechanism assumes:</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>injected code starts with new line - this makes ASI mechanism to finish user last statement if it was not properly finished by user, otherwise just creates</simpara> |
| </listitem> |
| <listitem> |
| <simpara>injected code does not have to be initiated manually - all exposed api is in named function declarations</simpara> |
| </listitem> |
| </orderedlist> |
| <simpara><emphasis>Explanation</emphasis></simpara> |
| <simpara>In first assumption we make workaround for user code that does not contain new line or semicolon at the end of last statement. This kind of code is incorrect and would result in last statement of user code and first statement of injected code to be interpreted as one JS statement. In most cases that would be invalid code. By having new line as first character of injected code, we are taking advantage of JS AutomaticSemicolonInjection mechanism. If user code AST is not finished properly this mechanism will finish close user AST. If user AST is finished properly, ASI will just insert empty statement between user code and injection code. In both cases we end up with proper AST.</simpara> |
| <simpara>Second assumption avoids need for further user code modifications, as injected does not have to be manually called. Instead we take advantage of variable and function hoisting mechanism of JS. This assures that even though user code is first in AST, JS environment will first initiate named functions therefore when user code calls injected code it is already defined in scope in which user code executes.</simpara> |
| </section> |
| <section xml:id="sec:Running_File_Code"> |
| <title>Running File Code</title> |
| <simpara>Second method of code execution is to execute provided file with user code. Normal way of doing that with NodeJS is to make call. But since we need to inject special code without rewriting files, we use different mechanism. Basically we are executing injected code and in the same scope using node api. Additionally we are attaching injected code to global scope in node, ensuring this way that required file is executed in scope which contains injected code. Putting this all together we are making following call:</simpara> |
| <simpara>This mechanism assumes that injected code attaches all exposed API to global scope .</simpara> |
| </section> |
| <section xml:id="sec:Injection_Code_Example"> |
| <title>Injection Code Example</title> |
| <simpara>Following is simple example of properly formed injection code.</simpara> |
| <programlisting language="n4js" linenumbering="unnumbered">; |
| function foo(){} |
| function bar(){} |
| function baz(){} |
| |
| (function(){ |
| GLOBAL.foo = foo; |
| GLOBAL.bar = bar; |
| GLOBAL.baz = baz; |
| })(); |
| ;</programlisting> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>first line is empty line to trigger ASI</simpara> |
| </listitem> |
| <listitem> |
| <simpara>second line (optional) enters</simpara> |
| </listitem> |
| <listitem> |
| <simpara>lines 3-5 are defining runtime api in current scope (in which user code provided as a string is executed)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>lines 6 (optional) is just a visual sugar</simpara> |
| </listitem> |
| <listitem> |
| <simpara>lines 7-11 are adding runtime api to global scope (to expose it when runnig user file with code)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>lines 12-13 (optional) are there to separate injected code and invokation of user file (if running user provided file with code)</simpara> |
| </listitem> |
| </orderedlist> |
| </section> |
| </section> |
| </chapter> |
| <chapter xml:id="_tests"> |
| <title>Tests</title> |
| <warning> |
| <simpara>This chapter may be outdated.</simpara> |
| </warning> |
| <simpara>In order to run all tests from command line, use maven:</simpara> |
| <screen>mvn clean verify</screen> |
| <simpara>You may have to increase the memory for maven via <literal>export MAVEN_OPTS="-Xmx2048m"</literal> (Unix) or <literal>set MAVEN_OPTS="-Xmx2048m"</literal> (Windows).</simpara> |
| <simpara>Do not run the tests via <literal>mvn clean test</literal> as this may lead to some failures.</simpara> |
| <section xml:id="sec:Performance_Tests" role="language-n4js"> |
| <title>Performance Tests</title> |
| <simpara>There are two kind of performance tests:</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>Synthetic Tests: an arbitrary number of test classes is generated, and then some modifications are performed on these classes.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Real World Tests: tests are based on a snapshot version of our platform libraries</simpara> |
| </listitem> |
| </orderedlist> |
| <section xml:id="sec:Synthetic_Performance_Tests"> |
| <title>Synthetic Performance Tests</title> |
| <simpara>The idea of the synthetic performance tests is to test the performance of specific functionality with a defined number classes, specially designed for the functionality under test.</simpara> |
| <simpara>The overall structure of the synthetic performance test is</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>generate test classes</simpara> |
| </listitem> |
| <listitem> |
| <simpara>compile these classes</simpara> |
| </listitem> |
| <listitem> |
| <simpara>modify the test classes</simpara> |
| </listitem> |
| <listitem> |
| <simpara>measure incremental build time</simpara> |
| </listitem> |
| </orderedlist> |
| <simpara>Step 3) and 4) can be done in a loop. Also, step 2) can be looped (with clean build).</simpara> |
| <simpara>The test classes are spread over clusters and projects. The following categories are used:</simpara> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Cluster</simpara> |
| </entry> |
| <entry> |
| <simpara>A cluster is a set of projects, each project of a cluster may depend on another project of the cluster. There are no dependencies between projects of different clusters</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Project</simpara> |
| </entry> |
| <entry> |
| <simpara>A project simply is a N4JS project, containing packages. A project may depend on other projects.</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Package</simpara> |
| </entry> |
| <entry> |
| <simpara>A package is a folder in a source folder of a project. A package contains classes.</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Class</simpara> |
| </entry> |
| <entry> |
| <simpara>A class is defined in a file, usually one class per file. The file, and with it the class, is contained in a package. The class contains members.</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Member</simpara> |
| </entry> |
| <entry> |
| <simpara>A member is either a field or method of a class. A method may has a body, which may contain variables with references to other classes.</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| <section xml:id="sec:Design_of_Generator" role="language-n4js"> |
| <title>Design of Generator</title> |
| <simpara><link linkend="fig:cd_performancetest_generator">Performance Generator</link> shows the classes of the performance test generator.</simpara> |
| <informalfigure xml:id="fig:cd_performancetest_generator"> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/15_tests/images/cd_performancetest_generator.svg"/> |
| </imageobject> |
| <textobject><phrase>cd performancetest generator</phrase></textobject> |
| </mediaobject> |
| </informalfigure> |
| <simpara>The package is designed as follows:</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara><literal>N4ProjectGenerator</literal> main control class for generation</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>TestDescriptor</literal> and subclasses: In order to <emphasis>keep memory consumption of the test class generator low</emphasis>, there is no graph structure created for the test elements. Instead, each element is uniquely named by a number, this number (actually a tuple of numbers) is stored in <literal>TestDescriptors</literal> and sub classes. There is a descriptor for each element of the tests.</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>AbstractModifier</literal> and subclasses generarate the tests. The idea is as follows:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><literal>Modifier</literal> generates all files, with complete references and no issues (complete)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>sub classes of <literal>Modifier</literal> skip certain generations or add modifications, leading to issues or solving them</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| </orderedlist> |
| <simpara>In order to compute the name of a class from its descriptor, as well as retrieving a class based on an absolute number, the modifiers use utility methods provided by <literal>PerformanceTestConfiguration</literal>. Note that computing the names and numbers depends on a configuration!</simpara> |
| </section> |
| <section xml:id="sec:Design_of_Performance_Test_Execution"> |
| <title>Design of Performance Test Configuration and Execution</title> |
| <simpara><xref linkend="fig:cd_performancetest_configAnRun"/> shows the classes of the performance test configuration and execution.</simpara> |
| <figure xml:id="fig:cd_performancetest_configAnRun"> |
| <title>Class Diagram of Performance Test Configuration and Execution</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/15_tests/images/cd_performancetest_configAnRun.svg"/> |
| </imageobject> |
| <textobject><phrase>cd performancetest configAnRun</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara>The package is designed as follows:</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara><literal>PerformanceTestConfiguration</literal> stores the test configuration. The configuration stores how many clusters, packages etc. are to be generated. It also provides methods for generating names from the descriptors mentioned above.</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>PerformanceMeter</literal> executes the test, listening to the (build) job to be finished etc.</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>AbstractGeneratingPerformanceTest</literal> Base test class contains setup, teardown and utility methods.</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>PerformanceTest</literal> Test class containing tests.</simpara> |
| </listitem> |
| </orderedlist> |
| </section> |
| <section xml:id="sec:JUnit_Configuration"> |
| <title>JUnit Configuration</title> |
| <simpara>We are using JUnitBenchamrks (<link xl:href="http://labs.carrotsearch.com/junit-benchmarks.html/">http://labs.carrotsearch.com/junit-benchmarks.html/</link>) to extend adjust plain JUnit behavior specifically to the performance tests needs.</simpara> |
| </section> |
| <section xml:id="sec:JUnitBenchmark_Test_Configuration"> |
| <title>JUnitBenchmark Test Configuration</title> |
| <simpara>JUnitBenchmark test configuration performed by annotating test method with <literal>@BenchmarkOptions</literal>. Parameters for that annotation include:</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara><literal>warmupRounds</literal> how many times test will be executed without taking measurement</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>benchmarkRounds</literal> how many times test will be executed, measurements taken will be used in results report</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>callgc</literal> Call <literal>System.gc()</literal> before each test. This may slow down the tests in a significant way.</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>concurrency</literal> specifies how many threads to use for tests.</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>clock</literal> specifies which clock to use.</simpara> |
| </listitem> |
| </orderedlist> |
| <simpara>Typical configuration for our performance tests might look like:</simpara> |
| <screen> @BenchmarkOptions(benchmarkRounds = 5, warmupRounds = 2) |
| @Test |
| public void test() throws Exception { |
| //test... |
| }</screen> |
| </section> |
| <section xml:id="sec:JUnitBenchmark_Report_Configuration"> |
| <title>JUnitBenchmark Report Configuration</title> |
| <simpara>By annotating TestClass in proper way, JUnitBenchamrks will generate html reports with performance results. There are two reports that can be generated:</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara><literal>@BenchmarkMethodChart</literal> report will contain results for every method from one test run (but all <literal>benchmarkRounds</literal> defined)</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><literal>filePrefix</literal> defines report file name</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara><literal>@BenchmarkHistoryChart</literal> report will contain trend of results from multiple test runs (it is aggregation of multiple instances of <literal>@BenchmarkMethodChart</literal> report)</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><literal>filePrefix</literal> defines report file name</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>labelWith</literal> defines label that will mark separate runs</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| </orderedlist> |
| <simpara><emphasis>labelWith</emphasis> property can have value propagated from run configuration/command line. example configuration might be <literal>@BenchmarkHistoryChart(filePrefix = benchmark-history, labelWith = LabelType.CUSTOM_KEY)</literal></simpara> |
| </section> |
| <section xml:id="sec:JUnitBenchmark_Run_Configuration"> |
| <title>JUnitBenchmark Run Configuration</title> |
| <simpara>It is possible to specify additional options for performance test run</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara><literal>-Djub.consumers=CONSOLE,H2</literal> specifies where results will be written, <emphasis>H2</emphasis> indicates H2 database to be used</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>-Djub.db.file=.benchmarks</literal> specifies name of the H2 database file</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>-Djub.customkey=</literal> value of that property scan be used as label in <literal>@BenchmarkHistoryChart</literal></simpara> |
| </listitem> |
| </orderedlist> |
| </section> |
| <section xml:id="sec:JUnitBenchmark_Example"> |
| <title>JUnitBenchmark Example</title> |
| <simpara>configuration example:</simpara> |
| <screen>@BenchmarkMethodChart(filePrefix = "benchmark-method") |
| @BenchmarkHistoryChart(filePrefix = "benchmark-history", labelWith = LabelType.CUSTOM_KEY) |
| public class PerformanceTest extends AbstractGeneratingPerformanceTest { |
| |
| public PerformanceTest() { |
| super("PerfTest"); |
| } |
| |
| @Rule |
| public TestRule benchmarkRun = new BenchmarkRule(); |
| |
| @Test |
| @BenchmarkOptions(benchmarkRounds = 5, warmupRounds = 2) |
| public void Test1() throws Exception { |
| |
| //Test... |
| } |
| |
| @Test |
| @BenchmarkOptions(benchmarkRounds = 5, warmupRounds = 2) |
| public void Test2() throws Exception { |
| |
| //Test... |
| } |
| }</screen> |
| <simpara>executing this code in Eclipse with configuration:</simpara> |
| <programlisting language="bash" linenumbering="unnumbered">-Xms512m -Xmx1024m -XX:MaxPermSize=512m $-$Djub.consumers=CONSOLE,H2 $-$Djub.db.file=.benchmarks $-$Djub.customkey=${current_date}</programlisting> |
| <simpara>will cause :</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>both tests to be executed 2 times for the warmup</simpara> |
| </listitem> |
| <listitem> |
| <simpara>both of tests being executed 5 times with measurement taken</simpara> |
| </listitem> |
| <listitem> |
| <simpara>results written to console</simpara> |
| </listitem> |
| <listitem> |
| <simpara>results stored in local H2 db file (created if doesn’t exist)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>generated <emphasis>benchmark-method.html</emphasis> with performance results of every test in that execution</simpara> |
| </listitem> |
| <listitem> |
| <simpara>generated <emphasis>benchmark-history.html</emphasis> with performance results of every execution</simpara> |
| </listitem> |
| <listitem> |
| <simpara>separate test executions will be labeled in <emphasis>benchmark-history.html</emphasis> with their start time</simpara> |
| </listitem> |
| </orderedlist> |
| </section> |
| <section xml:id="sec:Note_on_Jenkins_Job"> |
| <title>Note on Jenkins Job</title> |
| <simpara>For performance tests it is important not to get pass/fail result in terms of being below given threshold, but also to examine trend of those results. We achieve this by tooling described above. In order to keep this data independent of the build machine or build system storage, we are using separate repository to store performance artifacts. Jenkins in copying previous test results into workspace, runs performance tests, then commits and pushes combined results (adds current results to previous results) to repository.</simpara> |
| </section> |
| </section> |
| </section> |
| <section xml:id="sec:ECMA_Tests" role="language-n4js"> |
| <title>ECMA Tests</title> |
| <simpara>ECMAScript Language test262 is a test suite intended to check agreement between JavaScript implementations and ECMA-262, the ECMAScript Language Specification (currently 5.1 Edition).The test suite contains thousands of individual tests, each of which tests some specific requirements of the ECMAScript Language Specification. For more info refer to <link xl:href="http://test262.ecmascript.org/">http://test262.ecmascript.org/</link></simpara> |
| <simpara>Uses of this suite may include:</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>grammar tests</simpara> |
| </listitem> |
| <listitem> |
| <simpara>validation tests</simpara> |
| </listitem> |
| <listitem> |
| <simpara>run-time tests</simpara> |
| </listitem> |
| </orderedlist> |
| <simpara>ECMA test262 suite source code can be found here: <link xl:href="http://hg.ecmascript.org/tests/test262">http://hg.ecmascript.org/tests/test262</link></simpara> |
| <section xml:id="sec:Grammar_Tests"> |
| <title>Grammar Tests</title> |
| <simpara>Based on the JS files included in test262 suite we are generating tests that feed provided JS code into the parser. This operation will result in</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>parser throwing exceptions</simpara> |
| </listitem> |
| <listitem> |
| <simpara>parsed output will contain standard output</simpara> |
| </listitem> |
| </orderedlist> |
| <simpara>First case indicates that parsing provided JS code was not possible. This is considered to be Test Error.</simpara> |
| <simpara>Second case case indicates that parsing of the provided code was successful, and will result either</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>output with no errors - code adhered parser grammar</simpara> |
| </listitem> |
| <listitem> |
| <simpara>output with errors - code violated parser grammar</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>Given test must interpret those results to provide proper test output.</simpara> |
| <section xml:id="sec:Negative_Tests"> |
| <title>Negative Tests</title> |
| <simpara>It is important to note that some of the tests are positive and some are negative. Negative test cases are marked by the authors with <emphasis>@negative</emphasis> JSDoc like marker therefore parser tests must be aware of that to avoid both false positives and false negatives results.</simpara> |
| </section> |
| <section xml:id="sec:Test_Exclusion"> |
| <title>Test Exclusion</title> |
| <simpara>To exclude validation tests or run-time related test, implementation is blacklist approach to exclude some of the ECMA test262 tests from execution.</simpara> |
| </section> |
| </section> |
| </section> |
| <section xml:id="sec:Integration_Tests" role="language-n4js"> |
| <title>Integration Tests</title> |
| <simpara>Integration tests based on the stdlib and online-presence code bases can be found in bundle <literal>org.eclipse.n4js.hlc.tests</literal> |
| in package <literal>org.eclipse.n4js.hlc.tests.integration</literal> (headless case) and in bundle <literal>org.eclipse.n4js.ui.tests</literal> in |
| package <literal>org.eclipse.n4js.ui.tests.integration</literal> (plugin-UI tests running inside Eclipse). The headless |
| tests also execute mangelhaft tests, the UI tests only perform compilation of the test code.</simpara> |
| <simpara>More information can be found in the API documentation of classes <literal>AbstractIntegrationTest</literal> and <literal>AbstractIntegrationPluginUITest</literal>.</simpara> |
| </section> |
| <section xml:id="sec:Test_Helpers" role="language-n4js"> |
| <title>Test Helpers</title> |
| <simpara>Test helpers contain utility classes that are reused between different test plug-ins.</simpara> |
| <section xml:id="sec:Parameterized_N4JS_Tests"> |
| <title>Parameterized N4JS tests</title> |
| <simpara>Xtext JUnit test runer injects test a ParserHelper that allows to run N4JS parser on given input and obtain information abut parsing results. In some cases we want to run this kind of tests on large input data. To address this we provide two utilities ParameterizedXtextRunner and TestCodeProvider. They allow write data driven parser tests.</simpara> |
| <section xml:id="sec:ParameterizedXtextRunner"> |
| <title>ParameterizedXtextRunner</title> |
| <simpara>This This junit runner serves two purposes:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>injecting ParserHelper</simpara> |
| </listitem> |
| <listitem> |
| <simpara>creating multiple test instances for each input data provided</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>This class is based on @link org.eclipse.xtext.testing.XtextRunner and @link org.junit.runners.Parameterized</simpara> |
| </section> |
| <section xml:id="sec:TestCodeProvider"> |
| <title>TestCodeProvider</title> |
| <simpara>This class is repsonsible for extracting ZipEntry from provided ZipFile. Additinally it can filter out entries that match strings in provided black list file. Filtering out ZipEntries assumes that blacklist file contians Path of ZipEntry in ZipFile as string in one line. Lines starting with <emphasis>#</emphasis> in black list file are ignored by TestCodeProvider.</simpara> |
| </section> |
| <section xml:id="sec:Example_Of_Parameterized_Parser_Test"> |
| <title>Example of parameterized parser test</title> |
| <programlisting language="n4js" linenumbering="unnumbered">@RunWith(XtextParameterizedRunner.class) |
| @InjectWith(N4JSInjectorProvider.class) |
| public class DataDrivenParserTestSuite { |
| |
| /** |
| * Zip archives containing test files. |
| */ |
| public static final Collection<String> TEST_DATA_RESOURCES = Arrays.asList("foo.zip", "bar.zip"); |
| |
| /** |
| * Blacklist of files requiring an execution engine. |
| */ |
| public static final String BLACKLIST_FILENAME = "blacklist.txt"; |
| |
| /** |
| * Every generated test will use different ZipEntry as test data |
| */ |
| final ZipEntry entry; |
| |
| /** |
| * Name of resource containing corresponding ZipEntry |
| */ |
| final String resourceName; |
| |
| @Inject |
| protected ParseHelper<Script> parserN4JS; |
| |
| Collection<String> blackList; |
| |
| static final Logger logger = Logger.getLogger("someLogger"); |
| |
| public CopyOfLibraryParsingTestSuite(ZipEntry entry, String resourceName, Collection<String> blackList) { |
| this.entry = entry; |
| this.resourceName = resourceName; |
| this.blackList = blackList; |
| } |
| |
| @Rule |
| public TestRule blackListHandler = new TestRule() { |
| @Override |
| public Statement apply(final Statement base, Description description) { |
| final String entryName = entry.getName(); |
| if (blackList.contains(entryName)) { |
| return new Statement() { |
| @Override |
| public void evaluate() throws Throwable { |
| try { |
| base.evaluate(); |
| } catch (AssertionError e) { |
| // expected |
| return; |
| } |
| } |
| }; |
| } else { |
| return base; |
| } |
| } |
| }; |
| |
| /** |
| * Generates collection of ZipEntry instances that will be used as data |
| * provided parameter is mapped to name of the test (takes advantage of fact |
| * that ZipEntry.toString() is the same as entry.getName()) |
| * |
| * @return |
| * @throws URISyntaxException |
| * @throws ZipException |
| * @throws IOException |
| */ |
| @Parameters(name = "{0}") |
| public static Collection<Object[]> data() throws URISyntaxException, ZipException, IOException { |
| return TestCodeProvider.getDataFromZippedRoots(TEST_DATA_RESOURCES, BLACKLIST_FILENAME); |
| } |
| |
| /** |
| * generated instances of the tests will use this base implementation |
| * |
| * @throws Exception |
| */ |
| @Test |
| public void test() throws Exception { |
| assertNotNull(this.entry); |
| assertNotNull(this.resourceName); |
| assertNotNull(this.parserN4JS); |
| |
| //actual test code |
| |
| } |
| }</programlisting> |
| </section> |
| </section> |
| </section> |
| <section xml:id="sec:Issue_Suppression" role="language-n4js"> |
| <title>Issue Suppression</title> |
| <simpara>It can be useful to suppress certain issues before tests are ran, so that test expectations don’t have to consider inessential warnings. This means that the validator still returns a full list of issues, but before passing them to the testing logic, the issues are filtered.</simpara> |
| <simpara>When working with JUnit tests, the custom InjectorProvider <literal>N4JSInjectorProviderWithIssueSuppression</literal> can be used to configure them to suppress issues.</simpara> |
| <simpara>The codes that are suppressed are globally specified by the<?asciidoc-br?> |
| <literal>DEFAULT_SUPPRESSED_ISSUE_CODES_FOR_TESTS</literal> constant in <literal>N4JSLanguageConstants</literal>.</simpara> |
| <simpara>When working with Xpect tests, the XpectSetupFactory <literal>SuppressIssuesSetup</literal> can be used. See <link linkend="sec:Xpect_Issue_Suppression">Xpext Issue Suppression</link> for more details on Xpect issue suppression.</simpara> |
| </section> |
| <section xml:id="sec:Xpect_Tests" role="language-n4js"> |
| <title>Xpect Tests</title> |
| <simpara>For many tests, Xpect [<link linkend="Xpect">Xpect</link>] is used. Xpect allows for defining tests inside the language which is the language under test. That is, it is possible to refer to a JUnit test method in a special annotated comment, along with arguments passed to that method (typically expectations and the concrete location). Xpect comes with a couple of predefined methods which could be used there, e.g., tests for checking whether some expected error messages actually are produced. We have defined (and will probably define more) N4JS specific test methods.</simpara> |
| <simpara>In the following, we describe the most common Xpect test methods we use. Note that we do not use all types of tests shipped with Xpect. For example, AST tests (comparing the actual AST with an expected AST, using string dumps) is too hard to maintain.</simpara> |
| <simpara>Xpect test can be ignored by inserting a <literal>!</literal> between <literal>XPECT</literal> and the test name, e.g.</simpara> |
| <screen>// XPECT ! errors --> '~$message$~' at "~$location$~"</screen> |
| <section xml:id="sec:Xpect_Test_Setup"> |
| <title>Xpect Test Setup</title> |
| <simpara>The setup is either defined in the file itself, e.g.,</simpara> |
| <screen>/* XPECT_SETUP org.eclipse.n4js.spec.tests.N4JSSpecTest END_SETUP */</screen> |
| <simpara>or bundle-wide for a specific language in the plugin.xml (or fragment.xml), e.g.,</simpara> |
| <screen><extension point="org.xpect.testSuite"> |
| <testSuite class="org.eclipse.n4js.spec.tests.N4JSSpecTest" fileExtension="n4js" /> |
| </extension></screen> |
| </section> |
| <section xml:id="sec:Xpect_Issue_Suppression"> |
| <title>Xpect Issue Suppression</title> |
| <simpara>To configure an Xpect test class to suppress issues, you have to use the <literal>@XpectImport</literal> annotation to import the XpectSetupFactory <literal>org.eclipse.n4js.xpect.validation.suppression.SuppressIssuesSetup</literal>. Any Xpect test that is executed by this runner will work on the filtered list of issues.</simpara> |
| <simpara>Similar to issue suppressing JUnit tests, the suppressed issue codes are specified by<?asciidoc-br?> |
| <literal>DEFAULT_SUPPRESSED_ISSUE_CODES_FOR_TESTS</literal> constant in <literal>N4JSLanguageConstants</literal>.</simpara> |
| <simpara>For further per-file configuration a custom <literal>XPECT_SETUP</literal> parameter can be used. This overrides the suppression configuration of an Xpect runner class for the current file.</simpara> |
| <screen>/* XPECT_SETUP org.eclipse.n4js.tests.N4JSXpectTest |
| |
| IssueConfiguration { |
| IssueCode "AST_LOCAL_VAR_UNUSED" {enabled=true} |
| } |
| |
| END_SETUP |
| */</screen> |
| <simpara>In this example the issue code <literal>AST_LOCAL_VAR_UNUSED</literal> is explicitly enabled which means that no issue with this issue code will be suppressed.</simpara> |
| </section> |
| <section xml:id="sec:Xpect_Provided_Test_Methods"> |
| <title>Xpect Provided Test Methods</title> |
| <section xml:id="errors"> |
| <title>errors</title> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Definition</simpara> |
| </entry> |
| <entry> |
| <simpara>Single line:</simpara> |
| <screen>// XPECT errors --> '~$message$~' at "~$location$~"</screen> |
| <simpara>Multi line:</simpara> |
| <screen>/* XPECT errors --- |
| '~$message_1$~' at "~$location_1$~" |
| ~$\dots$~ |
| '~$message_n$~' at "~$location_n$~" |
| --- */</screen> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Description</simpara> |
| </entry> |
| <entry> |
| <simpara>Checks that one or more errors are issued at given location and compares the actual messages at a given location with the expected messages specified in the test.</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| <simpara>Also see <literal>no errors</literal> below.</simpara> |
| </section> |
| <section xml:id="warnings"> |
| <title>warnings</title> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Definition</simpara> |
| </entry> |
| <entry> |
| <simpara>Single line:<?asciidoc-br?></simpara> |
| <screen>// XPECT warnings --> '~$Message$~' at "~$Location$~"</screen> |
| <simpara>Multi line:</simpara> |
| <screen>/* XPECT warnings --- |
| '~$message_1$~' at "~$location_1$~" |
| ~$\dots$~ |
| '~$message_n$~' at "~$location_n$~" |
| --- */</screen> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Description</simpara> |
| </entry> |
| <entry> |
| <simpara>Checks that one or more warnings are issued at given location and compares the actual messages at a given location with the expected messages specified in the test.</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| </section> |
| <section xml:id="sec:N4JS_Specific_Xpect_Test_Methods"> |
| <title>N4JS Specific Xpect Test Methods</title> |
| <simpara>There are a lot of N4 specific Xpect tests methods available. To get all of these methods, search for references to annotation <literal>org.xpect.runner.Xpect</literal> in the N4 test plugins.</simpara> |
| <section xml:id="sec:XPECT_noerrors"> |
| <title>noerrors and nowarnings</title> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Definition</simpara> |
| </entry> |
| <entry> |
| <simpara>Single line:</simpara> |
| <screen>// XPECT noerrors --> '~$messageOrComment$~' at "~$location$~"</screen> |
| <simpara>Multi line:</simpara> |
| <screen>/* XPECT noerrors --- |
| '~$messageOrComment_1$~' at "~$location_1$~" |
| ~$\dots$~ |
| '~$messageOrComment_n$~' at "~$location_n$~" |
| --- */</screen> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Provided by</simpara> |
| </entry> |
| <entry> |
| <simpara><literal>NoerrorsValidationTest</literal></simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Description</simpara> |
| </entry> |
| <entry> |
| <simpara>Checks that at the given location <emphasis>no</emphasis> error (or warning) is issued. This tests is roughly speaker the opposite of <literal>errors</literal>. The idea behind this test is to replace comments in the code, stating that an expression is assumed to be valid, with an explicit test. This is in particular useful when you start working on a task, in which there are (wrong) errors at a given position, or for bug report.</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Example</simpara> |
| </entry> |
| <entry> |
| <informalexample> |
| <screen>function foo(any o): number { |
| if (o instanceof string) { |
| // XPECT noerrors --> "effect systems knows that o is a string" at "o" |
| return o.length; |
| } |
| return 0; |
| }</screen> |
| <simpara>is clearer and more explicit than</simpara> |
| <screen>function foo(any o): number { |
| if (o instanceof string) { |
| // here should be no error: |
| return o.length; |
| } |
| return 0; |
| }</screen> |
| <simpara>Also, the <literal>noerrors</literal> version will fail with a correct description, while the second one would fail with a general error and no location. Once the feature is implemented, regressions are detected much easier with the explicit version.</simpara> |
| </informalexample> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section xml:id="sec:XPECT_scope"> |
| <title>scope</title> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Definition</simpara> |
| </entry> |
| <entry> |
| <simpara>Single line:</simpara> |
| <screen>// XPECT scope at $location$ --> ~$[$~!~$]$~~$name_1$~, ~$\dots$~, ~$[$~!~$]$~~$name_n$~ ~$[$ ~, ...~$]$~</screen> |
| <simpara>Multi line:</simpara> |
| <screen>/* XPECT scope $location$ --- |
| ~$[$~!~$]$~~$name_1$~, ~$\dots$~, |
| ~$[$~!~$]$~~$name_n$~~$[$ ~, ...~$]$~ |
| --- */</screen> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Provided by</simpara> |
| </entry> |
| <entry> |
| <simpara><literal>PositionAwareScopingXpectTest</literal></simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Description</simpara> |
| </entry> |
| <entry> |
| <simpara>Checks that the expected elements are actually found in the scope (or explicitly not found, when <literal>!</literal> is used). This is a modified version of the Xpect built-in scope test, ensuring that also elements only put into the scope when they are explicitly requested are found.</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Example</simpara> |
| </entry> |
| <entry> |
| <screen>// XPECT scope at 'this.|$data_property_b' --> a, b, $data_property_b, !newB, ... |
| return this.$data_property_b + "_getter";</screen> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section xml:id="sec:XPECT_scopeWithPosition"> |
| <title>scopeWithPosition</title> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Definition</simpara> |
| </entry> |
| <entry> |
| <simpara>Single line:</simpara> |
| <programlisting language="n4js" linenumbering="unnumbered">// XPECT scopeWithPosition at $location$ --> ~$[$~!~$]$~~$name_1$~ - ~$pos_1$~, ~$\dots$~, ~$[$~!~$]$~~$name_n$~ - ~$pos_n$~ ~$[$ ~, ...~$]$~</programlisting> |
| <simpara>Multi line:</simpara> |
| <programlisting language="n4js" linenumbering="unnumbered">/* XPECT scopeWithPosition $location$ --- |
| ~$[$~!~$]$~~$name_1$~ - ~$pos_1$~, ~$\dots$~, |
| ~$[$~!~$]$~~$name_n$~ - ~$pos_n$~ ~$[$ ~, ...~$]$~ |
| --- */</programlisting> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Provided by</simpara> |
| </entry> |
| <entry> |
| <simpara><literal>PositionAwareScopingXpectTest</literal></simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Description</simpara> |
| </entry> |
| <entry> |
| <simpara>Checks that the expected elements are actually found in the scope (or explicitly not found, when <literal>!</literal> is used). The concrete syntax of the position, which is usually the line number, or the line number prefix with <literal>T</literal> if a type element is referenced, is described in <literal>EObjectDescriptionToNameWithPositionMapper</literal>.</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Example</simpara> |
| </entry> |
| <entry> |
| <screen>/* XPECT scopeWithPosition at foo2 --- |
| b - 9, |
| c - 25, |
| foo - T3, |
| foo2 - T9, |
| ... |
| ---*/ |
| foo2()</screen> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section xml:id="sec:XPECT_scopeWithResource"> |
| <title>scopeWithResource</title> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Definition</simpara> |
| </entry> |
| <entry> |
| <simpara>Single line:</simpara> |
| <screen>//</screen> |
| <simpara>Multi line:</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Provided by</simpara> |
| </entry> |
| <entry> |
| <simpara><literal>N4JSXpectTest</literal></simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Description</simpara> |
| </entry> |
| <entry> |
| <simpara>Compares scope including resource name but not line number.</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section xml:id="sec:XPECT_binding"> |
| <title>binding</title> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Definition</simpara> |
| </entry> |
| <entry> |
| <simpara>Single line:</simpara> |
| <screen>//</screen> |
| <simpara>Multi line:</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Provided by</simpara> |
| </entry> |
| <entry> |
| <simpara><literal>N4JSXpectTest</literal></simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Description</simpara> |
| </entry> |
| <entry> |
| <simpara>Checks that a given element is bound to something identified by (simple) qualified name. The check is designed as simple as possible. That is, simply the next following expression is tested, and within that we expect a property access or a direct identifiable element. The compared name is the simple qualified name, that is container (type) followed by elements name, without URIs of modules etc.</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section xml:id="sec:XPECT_linkedPathname"> |
| <title>linkedPathname</title> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Definition</simpara> |
| </entry> |
| <entry> |
| <simpara>Single line:</simpara> |
| <screen>// XPECT linkedPathname at '$location$' --> ~$pathname$~</screen> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Provided by</simpara> |
| </entry> |
| <entry> |
| <simpara><literal>LinkingXpectMethod</literal></simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Description</simpara> |
| </entry> |
| <entry> |
| <simpara>Checks that an identifier is linked to a given element identified by its path name. The path name is the qualified name in which the segments are separated by ’/’. This test does not use the qualified name provider, as the provider may return null for non-globally available elements. It rather computes the name again by using reflection, joining all <literal>name</literal> properties of the target and its containers.</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Example</simpara> |
| </entry> |
| <entry> |
| <screen>// XPECT linkedPathname at 'foo()' --> C/foo |
| new C().foo();</screen> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section xml:id="sec:XPECT_type_of"> |
| <title>type of</title> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Definition</simpara> |
| </entry> |
| <entry> |
| <simpara>Single line:</simpara> |
| <screen>// XPECT type of '$location$' --> ~$type$~</screen> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Provided by</simpara> |
| </entry> |
| <entry> |
| <simpara><literal>N4JSXpectTest</literal></simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Description</simpara> |
| </entry> |
| <entry> |
| <simpara>Checks that the type inferred at location is similar to expected type.</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Example</simpara> |
| </entry> |
| <entry> |
| <screen>// XPECT type of 'x' --> string |
| var x = 'hello'; |
| // XPECT type of 'foo()' --> union{A,number} |
| var any y = foo();</screen> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section xml:id="sec:XPECT_expectedType"> |
| <title>expectedType</title> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Definition</simpara> |
| </entry> |
| <entry> |
| <simpara>-</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Single line</simpara> |
| </entry> |
| <entry> |
| <screen>// XPECT expectedType at 'location' --&gt; Type</screen> |
| <literallayout class="monospaced">The location (at) is optional.</literallayout> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Provided by</simpara> |
| </entry> |
| <entry> |
| <simpara><literal>N4JSXpectTest</literal></simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Description</simpara> |
| </entry> |
| <entry> |
| <simpara>Checks that an element/expression has a certain expected type (i.e. Xsemantics judgment expectedTypeIn).</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section xml:id="sec:XPECT_elementKeyword"> |
| <title>elementKeyword</title> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Definition</simpara> |
| </entry> |
| <entry> |
| <simpara>Single line:</simpara> |
| <screen>// XPECT elementKeyword at 'myFunction' -> function</screen> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Example</simpara> |
| </entry> |
| <entry> |
| <screen>interface I { |
| fld: int; |
| get g(): string; |
| set s(p:string); |
| } |
| |
| //XPECT elementKeyword at 'string' --> primitive |
| var v1: string; |
| |
| //XPECT elementKeyword at 'I' --> interface |
| var i: I; |
| |
| //XPECT elementKeyword at 'fld' --> field |
| i.fld;</screen> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Provided by</simpara> |
| </entry> |
| <entry> |
| <simpara><literal>ElementKeywordXpectMethod</literal></simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Description</simpara> |
| </entry> |
| <entry> |
| <simpara>Checks that an element/expression has a certain element keyword. The expected element keyword is identical |
| to the element keyword shown when hovering the mouse over that element/expression in the N4JS IDE. This method is particuarly useful for testing merged elements of union/intersection.</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section xml:id="sec:XPECT_accessModifier"> |
| <title>accessModifier</title> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Definition</simpara> |
| </entry> |
| <entry> |
| <simpara>Single line:</simpara> |
| <screen>// XPECT accessModifier at 'myFunction' -> function</screen> |
| <simpara>or</simpara> |
| <screen>// XPECT accessModifier -> function</screen> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Example</simpara> |
| </entry> |
| <entry> |
| <screen>// XPECT accessModifier --> publicInternal |
| export @Internal public abstract class MyClass2 { |
| |
| // XPECT accessModifier --> project |
| abstract m1(): string; |
| |
| // XPECT accessModifier at 'm2' --> project |
| m2(): string { |
| return ""; |
| } |
| }</screen> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Provided by</simpara> |
| </entry> |
| <entry> |
| <simpara><literal>AccessModifierXpectMethod</literal></simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Description</simpara> |
| </entry> |
| <entry> |
| <simpara>Checks that an element/expression has a certain accessibility.</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section xml:id="sec:XPECT_compileResult"> |
| <title>compileResult</title> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Definition</simpara> |
| </entry> |
| <entry> |
| <simpara>Single line:</simpara> |
| <screen>//</screen> |
| <simpara>Multi line:</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Provided by</simpara> |
| </entry> |
| <entry> |
| <simpara>-</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Description</simpara> |
| </entry> |
| <entry> |
| <simpara><emphasis>This test should only be used during development of compiler and not used in the long run, because this kind of test is extremely difficult to maintain.</emphasis></simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section xml:id="sec:XPECT_output"> |
| <title>output</title> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Definition</simpara> |
| </entry> |
| <entry> |
| <simpara>Single line:</simpara> |
| <screen>//</screen> |
| <simpara>Multi line:</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Provided by</simpara> |
| </entry> |
| <entry> |
| <simpara>-</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Description</simpara> |
| </entry> |
| <entry> |
| <simpara>The most important test for compiler/transpiler, but also for ensuring that N4JS internal validations and assumptions are true at runtime.</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section xml:id="sec:XPECT_outputRegEx"> |
| <title>outputRegEx</title> |
| |
| </section> |
| <section xml:id="sec:XPECT_calculatedAccessModifier"> |
| <title>calculatedAccessModifier</title> |
| |
| </section> |
| <section xml:id="sec:XPECT_spec"> |
| <title>spec</title> |
| |
| </section> |
| <section xml:id="sec:XPECT_deadCode"> |
| <title>deadCode</title> |
| |
| </section> |
| <section xml:id="sec:XPECT_returnOrThrows"> |
| <title>returnOrThrows</title> |
| |
| </section> |
| <section xml:id="sec:XPECT_lint"> |
| <title>lint</title> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Definition</simpara> |
| </entry> |
| <entry> |
| <simpara>Single line:</simpara> |
| <screen>/* XPECT lint */</screen> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Provided by</simpara> |
| </entry> |
| <entry> |
| <simpara><literal>CompileAndLintTest</literal></simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Description</simpara> |
| </entry> |
| <entry> |
| <simpara>Passes the generated code through the JSHint JavaScript linter. This test includes for instance checking for missing semicolons and undefined variables. The whole test exclusively refers to the generated javascript code.</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section xml:id="sec:XPECT_lintFails"> |
| <title>lintFails</title> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Definition</simpara> |
| </entry> |
| <entry> |
| <simpara>Single line:</simpara> |
| <screen>/* XPECT lintFails */</screen> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Provided By</simpara> |
| </entry> |
| <entry> |
| <simpara><literal>CompileAndLintTest</literal></simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Description</simpara> |
| </entry> |
| <entry> |
| <simpara>Negation of lint. Fails on linting success. Expects linting errors.</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| </section> |
| <section xml:id="sec:FIXME_Xpect_modifier"> |
| <title>FIXME Xpect modifier</title> |
| <simpara>A modification of the official Xpect framework allows us to use the FIXME modifier on each test. <footnote><simpara>Currently we use our own fork of Xpect <link xl:href="https://github.com/NumberFour/Xpect">https://github.com/NumberFour/Xpect</link> and the respective p2-repository <link xl:href="https://numberfour.github.io/Xpect/updatesite/nightly/">https://numberfour.github.io/Xpect/updatesite/nightly/</link></simpara></footnote></simpara> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Syntax</simpara> |
| </entry> |
| <entry> |
| <simpara>FIXME can be applied on any test just after the XPECT keyword:</simpara> |
| <screen>// XPECT FIXME xpectmethod ... --> ...</screen> |
| <simpara>Tests will still be ignored if an exclamation mark (!) is put between XPECT and FIXME.</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Description</simpara> |
| </entry> |
| <entry> |
| <simpara>Using FIXME on a test negates the result of the underlying JUnit test framework. Thus a failure will be reported as a <literal>true assertion</literal> and an assertion that holds will be reported as <literal>failure</literal> . This enables to author valuable tests of behaviour, which is still not functional.</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Example</simpara> |
| </entry> |
| <entry> |
| <simpara>For instance, if we encounter an error-message at a certain code position, but the code is perfectly right, then we have an issue. We can annotate the situation with a ’fix me’ ’noerrors’ expectation:</simpara> |
| <screen>// Perfectly right behaviour XPECT FIXME noerrors --> |
| console.log("fine example code with wrong error marker here.");</screen> |
| <simpara>This turns the script into an Xpect test. We can integrate the test right away into our test framework and it will not break our build (even though the bug is not fixed).</simpara> |
| <simpara>When the issue will be worked on, the developer starts with removing ’FIXME’ turning this into useful unit-test.</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| <simpara>It is crucial to understand that FIXME negates the whole assertion. Example: If one expects an error marker at a certain position using the ’errors’ directive, one must give the exact wording of the expected error-message to actually get the FIXME behaviour working. To avoid strange behaviour it is useful to describe the expected error a comment in front of the expectation and leave the message-section empty.</simpara> |
| </section> |
| <section xml:id="sec:Expectmatrix_Xpect_Test_Methods"> |
| <title>Expectmatrix Xpect tests</title> |
| <simpara>Applying test-driven development begins with authoring acceptance and functional tests for the work in the current sprint. By this the overall code quality is ensured for the current tasks to solve. Rerunning all collected tests with each build ensures the quality of tasks solved in the past. Currently there is no real support for tasks, which are not in progress but are known to be processed in the near or far future. Capturing non-trivial bug reports and turning them into reproducable failing test-cases is one example.</simpara> |
| <simpara>Usually people deactivate those future-task-tests in the test code by hand. This approach doesn’t allow to calculate any metrics about the code. One such metric would be: Is there any reported bug solved just by working on an (seemingly unrelated) scheduled task?</simpara> |
| <simpara>To achieve measurements about known problems, a special build-scenario is set up. As a naming convention all classes with names matching <literal>* Pending</literal> are assumed to be Junit tests. In bundle <literal>org.eclipse.n4js.expectmatrix.tests</literal> two different Xpect-Runners are provided, each working on its own subfolder. Usual Xpect-tests are organised in folder xpect-test while in folder xpect-pending all future-tests are placed. A normal maven-build processes only the standard junit and xpect tests. Starting a build with profile <literal>execute-expectmatrix-pending-tests</literal> will additionally execute Xpect tests from folder xpect-pending and for all bundles inheriting from <literal>/tests/pom.xml</literal> all unit tests ending in <literal>* Pending</literal>. This profile is deactivated by default.</simpara> |
| <simpara>A special jenkins-job - N4JS-IDE_nightly_bugreports_pending - is configured to run the pending tests and render an overview und history to compare issues over time. Due to internal Jenkins structures this build always marked failed, even though the maven-build succeeds successfully.</simpara> |
| <simpara>Relevant additional information can be found in</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>Jenkins job for pending cases: <link xl:href="http://build-master:8080/view/N4JS/job/N4JS-IDE_nightly_bugreports_pending/">http://build-master:8080/view/N4JS/job/N4JS-IDE_nightly_bugreports_pending/</link></simpara> |
| </listitem> |
| <listitem> |
| <simpara>Testmatrix <link xl:href="https://docs.google.com/a/numberfour.eu/spreadsheets/d/1Blo58cRwIWemaiBNSnOtsQ8U7b3FoodX3yEs7oJKIg0/">https://docs.google.com/a/numberfour.eu/spreadsheets/d/1Blo58cRwIWemaiBNSnOtsQ8U7b3FoodX3yEs7oJKIg0/</link></simpara> |
| </listitem> |
| </itemizedlist> |
| </section> |
| <section xml:id="xpect-lint-tests"> |
| <title>Xpect Lint Tests</title> |
| <figure xml:id="sec:XPECT_Lint_Tests"> |
| <title>Xpect Lint</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/15_tests/images/diag_XpectLint.svg"/> |
| </imageobject> |
| <textobject><phrase>diag XpectLint</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara>The test transpiles the provided n4js resource and checks the generated code. This is achieved using the Javascript linter JSHint.</simpara> |
| <simpara>After transpiling the provided n4js resource the LintXpectMethod combines the generated code with the jshint code into a script. It calls the JSHint validation function and returns the linting result as a json object. The error results are displayed in the console. The script is executed using the <literal>Engine</literal> class. (<link linkend="sec:Design">Design</link>)</simpara> |
| <simpara>For the linting process an adapted configuration for JSHint is used. For the needs of N4JS the linter is configured to recognise N4JS specific globals. Details about the error codes can be found at the <link xl:href="https://github.com/jshint/jshint/blob/2444a0463e1a99d46e4afa50ed934c317265529d/src/messages.js">jshint repository</link>.</simpara> |
| <simpara>The following warnings are explicitly enabled/disabled:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><emphasis role="strong">W069</emphasis>: [’a’] is better written in dot notation <emphasis role="strong">DISABLED</emphasis></simpara> |
| </listitem> |
| <listitem> |
| <simpara><emphasis role="strong">W033</emphasis>: Missing semicolon <emphasis role="strong">ENABLED</emphasis></simpara> |
| </listitem> |
| <listitem> |
| <simpara><emphasis role="strong">W014</emphasis>: Bad line breaking before ’a’. <emphasis role="strong">DISABLED</emphasis></simpara> |
| </listitem> |
| <listitem> |
| <simpara><emphasis role="strong">W032</emphasis>: Uneccesarry semicolon <emphasis role="strong">ENABLED</emphasis></simpara> |
| </listitem> |
| <listitem> |
| <simpara><emphasis role="strong">W080</emphasis>: It’s not necessary to initialize ’a’ to ’undefined’. <emphasis role="strong">ENABLED</emphasis></simpara> |
| </listitem> |
| <listitem> |
| <simpara><emphasis role="strong">W078</emphasis>: Setter is defined without getter. <emphasis role="strong">DISABLED</emphasis></simpara> |
| </listitem> |
| <listitem> |
| <simpara>ES6 related warnings are <emphasis role="strong">disabled</emphasis> using the ’esnext’ option: <emphasis role="strong">W117</emphasis>: Symbol is not defined. <emphasis role="strong">DISABLED</emphasis> <emphasis role="strong">W104</emphasis>: ’yield’ is only available in ES6 <emphasis role="strong">DISABLED</emphasis> <emphasis role="strong">W117</emphasis>: Promise is not defined <emphasis role="strong">DISABLED</emphasis> <emphasis role="strong">W119</emphasis>: function* is only available in ES6 <emphasis role="strong">DISABLED</emphasis></simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>The xpect lint test only applies if the provided resource passes the n4js compiler.</simpara> |
| <simpara>The xpect method lintFails can be used to create negative tests. All linting issues discovered during the development of the xpect plugin have there own negative test to keep track of their existence.</simpara> |
| <simpara>Additional information:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>JSHint: <link xl:href="http://jshint.com/docs/">http://jshint.com/docs/</link></simpara> |
| </listitem> |
| </itemizedlist> |
| </section> |
| </section> |
| <section xml:id="xpect-proposal-tests" role="language-n4js"> |
| <title>Xpect Proposal Tests</title> |
| <simpara>Proposal tests are all tests which verify the existence and application of completion proposals, created by content assist, quick fixes etc.</simpara> |
| <simpara>The key attributes of a proposal (cf <literal>ConfigurableCompletionProposal</literal>) are:</simpara> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>displayString</simpara> |
| </entry> |
| <entry> |
| <simpara>the string displayed in the proposal list</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>replacementString</simpara> |
| </entry> |
| <entry> |
| <simpara>simple variant of which is to be added to document, not necessarily the whole replacement (as this may affect several locations and even user interaction)</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| <simpara>In the tests, a <emphasis>proposal is identified by a string contained in the displayString</emphasis>. If several proposal match, test will fail (have to rewrite test setup or proposal identifier to be longer). Proposal identifier should be as short as possible (to make test robust), but not too short (to make test readable).</simpara> |
| <simpara>The following proposal tests are defined:</simpara> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>contentAssist <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mo>[</mo></mrow></math> List <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mo>]</mo></mrow></math></simpara> |
| </entry> |
| <entry> |
| <simpara>verifies proposals created by content assist</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>quickFix <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mo>[</mo></mrow></math> List <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mo>]</mo></mrow></math></simpara> |
| </entry> |
| <entry> |
| <simpara>verifies proposals created by quick fixes. Cursor position is relevant, that’s handled by the framework. We only create tests with cursor position – fixes applied via the problem view should work similarly, but without final cursor position.</simpara> |
| <simpara>If no error is found at given position, test will fail (with appropriate error message!). In call cases of apply, the issue must be resolved. Usually, fixing an issue may leave the file invalid as other issues still exists, or because by fixing one issue others may be introduced (which may happen often as we try to avoid consequential errors in validation). For some special cases, quickFix tests support special features, see below.</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| <simpara>Not tested in this context: Verify proposal description, as these tests would be rather hard to maintain and the descriptions are often computed asynchronously.</simpara> |
| <section xml:id="sec:Validation_vs__Non_Validation"> |
| <title>Validation vs. Non-Validation</title> |
| <simpara>We expect proposal tests to be applied on non-valid test files, and usually file is also broken after a proposal has been applied. Thus, the test-suite must not fail if the file is not valid.</simpara> |
| </section> |
| <section xml:id="sec:General_Proposal_Test_Features"> |
| <title>General Proposal Test Features</title> |
| <section xml:id="sec:Test_Variables"> |
| <title>Test Variables</title> |
| <simpara>Often, list of proposals are similar for different tests (which define different scenarios in which the proposals should be generated). For that reason, variables can be defined in the test set up:</simpara> |
| <simpara>In the Xpect-Setup there is now a special <literal>Config</literal> component where specific switches are accessible. For instance the timeout switch for content assist can be modified:</simpara> |
| <screen>/* XPECT_SETUP org.eclipse.n4js.tests.N4JSNotValidatingXpectPluginUITest |
| ... |
| Config { |
| content_assist_timeout = 1000 |
| ... |
| } |
| ... |
| */</screen> |
| <simpara>Note: There should only be one <literal>Config</literal> component per Xpect-Setup.</simpara> |
| <simpara>Variables are introduced via the <literal>VarDef</literal> component. It takes a string argument as the variable name on construction. Inside the body one add <literal>MemberLists</literal> and <literal>StringLists</literal> arguments. Variable definitions may appear in <literal>Config</literal> bodies or in the Xpect-Setup.</simpara> |
| <screen>VarDef "objectProposals" { |
| ... |
| }</screen> |
| <simpara>Define variables with expression: A simple selector is given with the <literal>MemberList</literal> component. These components take three <literal>String</literal> arguments in the constructor. The first one is a typename. The second one is the feature selector, e.g. <literal>methods</literal> , <literal>fields</literal> , …and the third one defines the visibility.</simpara> |
| <screen>/* XPECT_SETUP |
| VarDef "stringProposals" { MemberList "String" "methods" "public" {}} |
| END_SETUP */</screen> |
| <simpara>We have to define a filter later in Xtend/Java, e.g., <literal>getClassWithName( <emphasis>className</emphasis> ).filterType(<emphasis>methods</emphasis>).filterVisibility(<emphasis>accessodifier</emphasis>)…​</literal></simpara> |
| <simpara>A variable is later referenced as follows:</simpara> |
| <screen><$variable></screen> |
| <simpara>Usage example:</simpara> |
| <screen>// XPECT contentAssistList at 'a.<|>methodA' proposals --> <$stringProposals>, methodA2 |
| a.methodA</screen> |
| </section> |
| <section xml:id="sec:Location_and_Selection"> |
| <title>at – Location and Selection</title> |
| <simpara>Tokens in expectation/setup:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><literal><|></literal> cursor position</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal><[></literal> selection start → also defines cursor position</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal><]></literal> selection end</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>All proposal tests have to specify a location via <literal>at</literal>, the location must contain the cursor position and may contain a selection. E.g.:</simpara> |
| <screen>// XPECT contentAssistList at 'a.<|>methodA' apply 'methodA2' --> a.methodA2()<|>methodA |
| // XPECT contentAssistList at 'a.<|><[>methodA<]>' apply 'methodA2' override --> a.methodA2()<|> |
| a.methodA</screen> |
| </section> |
| <section xml:id="sec:Multi_Line_Expectations_in_Proposal_Tests"> |
| <title>Multi Line Expectations in Proposal Tests</title> |
| <simpara>In multiline expectations, ignored lines can be marked with <literal><…​></literal>. This means that 0 to n lines may occur but are ignored for comparison.</simpara> |
| <simpara>All multiline expectations are compared line-wise, with exact match except line delimiters (which are ignored as well)</simpara> |
| </section> |
| <section xml:id="sec:Timeout"> |
| <title>Timeout and Performance</title> |
| <simpara>We define a default timeout for content assist tests. In set up, this timeout may be changed:</simpara> |
| <programlisting language="n4js" linenumbering="unnumbered">/* XPECT SETUP |
| ... |
| content_assist_timeout = 2000ms |
| ... |
| END_SETUP */</programlisting> |
| <simpara>Performance is measured by measuring the runtime of tests, we should later setup performance measurements similar to the performance tests.</simpara> |
| </section> |
| </section> |
| <section xml:id="proposals-verify-existence-of-proposals"> |
| <title>proposals – Verify Existence of Proposals</title> |
| <simpara>In general, one could verify if certain proposals are present or not present, and in which order they are present. This is verified by the <literal>proposals</literal> argument.</simpara> |
| <simpara>E.g.</simpara> |
| <screen>// XPECT contentAssistList at 'a.<|>methodA' proposals --> <$stringProposals>, methodA2 |
| a.methodA</screen> |
| <simpara>Additional flags:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>Order modifier:</simpara> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>unordered</simpara> |
| </entry> |
| <entry> |
| <simpara>by default</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>ordered</simpara> |
| </entry> |
| <entry> |
| <simpara>the given expectations have to have that order, between these expectations other proposals may be present (in case of contains)</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </listitem> |
| <listitem> |
| <simpara>Subset modifier:</simpara> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>exactly</simpara> |
| </entry> |
| <entry> |
| <simpara>(default, maybe changed later) no other expectations as the given ones (usually <literal>contains</literal> is recommended).</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>contains</simpara> |
| </entry> |
| <entry> |
| <simpara>at least the given expectations must be present, but others may be there as well.</simpara> |
| <simpara>Using <emphasis>contains</emphasis> must be used with care since we match items by searching for a proposal which contains one of the expected strings as a substring. So if the only available proposal were ’methodA2’ and we would test if proposals contain ’methodA’, ’methodA2’ we would obtain a passing test.</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>not</simpara> |
| </entry> |
| <entry> |
| <simpara>any of the given proposals must be NOT be proposed</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </listitem> |
| </itemizedlist> |
| </section> |
| <section xml:id="sec:Verify_displayed_string"> |
| <title>display – Verify displayed string</title> |
| <simpara>We do not verify the text style. We only verify text:</simpara> |
| <screen>// XPECT contentAssistList at 'a.<|>methodA' display 'methodA2' --> 'methodA2(): any - A' |
| a.methodA</screen> |
| <simpara>This kind of test should be only applied for few scenarios, because the long display tests are rather hard to maintain.</simpara> |
| </section> |
| <section xml:id="sec:Apply_Proposal"> |
| <title>apply – Apply Proposal</title> |
| <simpara>Execution of proposal, the expectation describes the expected text result. The tests follow the naming convention of Ending in …Application.</simpara> |
| <simpara>Additional flags:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>insertion mode</simpara> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>insert</simpara> |
| </entry> |
| <entry> |
| <simpara>(default) Ctrl not pressed, proposal is inserted at given location</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>override</simpara> |
| </entry> |
| <entry> |
| <simpara>Ctrl is pressed, proposal overrides selection.</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </listitem> |
| </itemizedlist> |
| <simpara>Single line assumes change at line of initial cursor position:</simpara> |
| <screen>// XPECT contentAssist at 'a.<|>methodA' apply 'methodA2' --> a.methodA2()methodA |
| // XPECT contentAssist at 'a.<|>methodA' apply 'methodA2' insert --> a.methodA2()methodA |
| // XPECT contentAssist at 'a.<|>methodA' apply 'methodA2' override --> a.methodA2() |
| a.methodA</screen> |
| <simpara>or</simpara> |
| <screen>// XPECT quickFix at 'a.<|>method' apply 'methodA2' --> a.methodA2(); |
| a.methodTypo();</screen> |
| <simpara>Multiline expectations describe changes to the whole. In order to match the expectation context information around the relevant places must be given. The format is similar to a unified diff with a special rule for inline-changes. The comparison works in a line-based mode. Each line in the expectation is prefixed with one character of ’+’, ’|’, ’-’ or ’ ’. Inserted lines are marked with + and removed lines with -. Lines marked with | denote changes in the line. Difference is placed inline inside of a pair of square brackets with a | separating the removal on the left and the addition on the right.</simpara> |
| <screen>/* XPECT contentAssistList at 'a.me<|>thodA' apply 'methodA2' --- |
| <...> |
| import A from "a/b/c" |
| <...> |
| a.methodA2()<|>methodA |
| <...> |
| --- */ |
| a.methodA</screen> |
| <screen>/* XPECT contentAssistList at 'a.me<|>thodA' apply 'methodA2' --- |
| import B from "p/b" |
| +import A from "a/b/c" |
| |
| ... |
| foo() { |
| |a.[me|thodA2()] |
| } |
| ... |
| --- */ |
| a.methodA</screen> |
| <section xml:id="resource-application-in-other-files"> |
| <title>resource – application in other files</title> |
| <simpara>The resource parameter is available for the quickfix xpect method. It specifies the target resource of the quickfix. (e.g. change declaration of type in another file to quickfix an issue).</simpara> |
| <screen>/* XPECT quickFix at 'sameVendor.protected<|>Method()' apply 'Declare member as public, @Internal' resource='../../otherProject/src/other_file.n4js' --- |
| diff here |
| --- |
| */</screen> |
| <simpara>The syntax is similar to a normal multiline quickfix xpect test besides the addition of the resource parameter. The relative path points to a file in the same workspace as the test file. Note that the path refers to the folder structure specified in the XPECT SETUP header. It is relative to the folder the test file is contained in.<?asciidoc-br?> |
| <?asciidoc-br?> |
| The diff is between the specified resource before and after the application of the quickfix<?asciidoc-br?> |
| <?asciidoc-br?> |
| Note that the fileValid (<link linkend="fileValidVerify-validation-status">Verify Validation Status</link>) parameter is not applicable to an extern resource.</simpara> |
| </section> |
| </section> |
| <section xml:id="sec:Content_Assist_Cycling"> |
| <title>kind – Content Assist Cycling</title> |
| <simpara>We assume the default kind to be ’n4js’. It is possible to select a proposal kind in the test set up or via the argument <literal>kind</literal> in the test.</simpara> |
| <simpara>Select kind in test setup:</simpara> |
| <screen>/* XPECT_SETUP |
| content_assist_kind = 'recommenders' |
| END_SETUP */</screen> |
| <simpara>Select kind in test:</simpara> |
| <screen>// XPECT contentAssistList kind 'smart' at 'a.<|>methodA' display 'methodA2' --> 'methodA2(): any - A' |
| a.methodA</screen> |
| </section> |
| <section xml:id="fileValidVerify-validation-status"> |
| <title>fileValid – Verify validation status</title> |
| <simpara>In some cases, in particular in case of quick fix tests, the file should be valid after the proposal has been applied. This is added by an additional argument <literal>fileValid</literal>.</simpara> |
| <simpara>E.g.,</simpara> |
| <screen>// XPECT quickFix at 'a.<|>method' apply 'methodA2' fileValid --> a.<|>methodA2(); |
| a.methodTypo();</screen> |
| </section> |
| </section> |
| <section xml:id="sec:Apply_Proposal_And_Execute_Tests" role="language-n4js"> |
| <title>Apply Proposal And Execute Tests</title> |
| <simpara>If a proposal fixes all issues in a test file, the file could be executed. This is an important type of test, as this is what the user expects in the end. Besides, this type of test is very robust, as it does not depend on the concrete way how an issue is fixed. For quick fixes, these kind of tests are to be implemented!</simpara> |
| <simpara>The following combined proposal and execute tests are provided:</simpara> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>quickFixAndRun</simpara> |
| </entry> |
| <entry> |
| <simpara>applies a quick fix, verifies that all issues are solved, and executes the file.</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| <simpara>These tests are basically execution tests, that is the expectation describes the expected output of the script.</simpara> |
| <simpara>E.g.</simpara> |
| <screen>// XPECT quickFixAndRun at 'a.<|>method' apply 'methodHelloWorld' --> Hello World |
| a.methodTypo();</screen> |
| <simpara>with <literal>methodHelloWorld</literal> printing <literal>’Hello World’</literal> to the console. The expected output can be multiline:</simpara> |
| <screen>/* XPECT quickFixAndRun at 'a.<|>method' apply 'methodHelloWorld' --- |
| Hello World |
| --- */ |
| a.methodTypo();</screen> |
| </section> |
| <section xml:id="sec:Organize_Imports_Test" role="language-n4js"> |
| <title>Organize Imports Test</title> |
| <simpara>For testing organise imports a Plugin-UI test method is available. It operates in two modes. Either a successful application of organise imports is tested or the expected abortion is checked.</simpara> |
| <section xml:id="organizeimports"> |
| <title>organizeImports</title> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Definition</simpara> |
| </entry> |
| <entry> |
| <simpara>Multi line:</simpara> |
| <screen>/* XPECT organizeImports --- |
| ~Failure given by line-syntax starting with two characters:~ |
| + additional line |
| | line with inplace[removed part|added part] |
| + removed line |
| unchanged line |
| --- */ |
| // XPECT warnings --> "The import of A is unused." at "A" |
| import A from "a/a1/A" |
| ... |
| }</screen> |
| <simpara>Single line:</simpara> |
| <screen>// XPECT organizeImports ambiguous '~Failure String of Exception~'--> |
| }</screen> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Provided by</simpara> |
| </entry> |
| <entry> |
| <simpara><literal>OrganizeImportXpectMethod</literal></simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Description</simpara> |
| </entry> |
| <entry> |
| <simpara>Checks that the resulting source-file differs in the described way. The Multiline variant checks the result of a successful application of organise import to the file. All errors and warnings prior to organising imports must be marked as the appropriate XPECT-error or warning.</simpara> |
| <simpara>The single-line notation checks the failure of an fully automatic reorganisation of the imports due to some reason. The reason is verified to be the given string after the <literal>ambiguous</literal> keyword. The expectation side (right of <literal>-→</literal>) is empty.</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Example</simpara> |
| </entry> |
| <entry> |
| <screen>/* XPECT_SETUP org.eclipse.n4js.tests.N4JSXpectPluginUITest |
| Workspace { |
| Project "P1" { |
| Folder "src" { |
| Folder "a" { Folder "a1" { File "A.n4js" { from="../../a/a1/A.n4js" } } |
| Folder "c" { ThisFile {} } } } |
| File "package.json" { from="package_p1.json" } } } |
| END_SETUP |
| */ |
| |
| /* XPECT organizeImports --- |
| |
| | import [A from "a/a1/A"|] |
| | [import|] AR from "a/a1/A" |
| export public role BRole with AR { |
| } |
| --- */ |
| // XPECT warnings --> "The import of A is unused." at "A" |
| import A from "a/a1/A" |
| import AR from "a/a1/A" |
| |
| // XPECT noerrors --> "Couldn't resolve reference to Type 'AR'." |
| export public role BRole with AR { |
| }</screen> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| </section> |
| <section xml:id="sec:Access_Control_Test" role="language-n4js"> |
| <title>Access Control Test</title> |
| <simpara>Access control refers to the decision whether or not a member or a top level element is accessible for a client. In this context, access refers to reading, writing, and calling a member or a top level function, and to overriding inherited members in classes and interfaces. In N4JS, access is controlled via modifiers at the type and at the member level. Due to the large number of such modifiers and the large number of different scenarios for access control, manually written tests can only cover a small number of actual scenarios. An automatic test generator helps to increase the test coverage for access control.</simpara> |
| <simpara>The test generator loads test scenarios from a CSV table that also contains the expected results of each test case and then generates N4JS projects and code for each test case, compiles them using the headless compiler, and compares the compilation results to the expectations from the CSV table. Note that the test generator does not generate test cases that check access control for top level functions and variables.</simpara> |
| <section xml:id="test-scenarios"> |
| <title>Test Scenarios</title> |
| <simpara>Each test scenario consists of a scenario specifier (one of <literal>Extends</literal> , <literal>Implements</literal> , <literal>References</literal> ), a supplier and a client (each of which can be a class, and abstract class, an interface, or an interface with default implementations of its getters, setters, and methods). Thereby, the client attempts to access a member of the supplier either by reading, writing, or calling it or by overriding it in the case of an <literal>Extends</literal> or <literal>Implements</literal> scenario. Furthermore, each test cases specifies the location of the client in relation to the location of the supplier, e.g., whether the client is in the same module, or whether it is in the same project (but not the same module), and so forth. Finally, a test case must specify the access control modifiers of the supplier type and the member that is being accessed by the client, whether that member is static or not, and, lastly, the type of access that is being attempted.</simpara> |
| </section> |
| <section xml:id="n4js-code-generator"> |
| <title>N4JS Code Generator</title> |
| <simpara>The test generator needs to generate N4JS projects and modules that implement a particular test case. For this purpose, it uses a simple code generator, available in the package <literal>org.eclipse.n4js.tests.codegen</literal>. The classes in that package allow specifying N4JS projects, modules, classes, and members in Java code and to generate the required artifacts as files at a given location.</simpara> |
| </section> |
| <section xml:id="xtext-issue-matcher"> |
| <title>Xtext Issue Matcher</title> |
| <simpara>To match the test expectations from the CSV file against the issues generated by the compiler, the test generator uses a small library of issue matchers, available in the package <literal>org.eclipse.n4js.tests.issues</literal>. The classes in that package make it easy to specify expectations against Xtext issues programmatically and to evaluate these expectations against a specific set of Xtext issues. Mismatches, that is, unexpected issues as well as unmatched expectations, are explained textually. These explanations are then shown in the failure notifications of the test case generator.</simpara> |
| <simpara>The test expectations allow for FIXME annotations in the CSV table to express cases where an access control error has been found, but hasn’t been fixed yet. Such expectations lead to inverted expectations against the generated issues: For example, if a test expects an attempted access to fail with an error, but the validation rules allow the access regardless, then a FIXME annotation at the test will invert the actual expectation: Where previously an error was expected, there must now be no error at all. Then, once the validation rules have been adjusted and the error occurs as expected, the FIXME test case will fail until the FIXME annotation has been removed. Therefore, the issue expectation matchers can be inverted and adjust their behavior accordingly.</simpara> |
| </section> |
| </section> |
| <section xml:id="sec:Smoke_Tests" role="language-n4js"> |
| <title>Smoke Tests</title> |
| <simpara>Smoke tests are useful to ensure the robustness of the IDE. They aim at revealing potential exceptions that may surface to the end user in the IDE or in a headless compile run. Therefore, different permutations of a given input document are fed into processing components such as the validator or the type system. No exceptions may be thrown from these components.</simpara> |
| <simpara>Since smoke tests are longrunning, it is not desired to execute them with each Maven run. That’s why the naming convention <literal>* Smoketest</literal> was choosen. It will not be matched by the naming pattern for normal JUnit tests during a maven run.</simpara> |
| <simpara>The smoke tests are generally created by using the (valid or invalid) input of existing test cases. Therefore, the a command line argument <literal>-DSmokeTestWriter=true</literal> may be passed to the VM that executes a test. All input documents that are processed by a <literal>ParseHelper<Script></literal> will be written to a new test method on the console. The result can be merged manually into the <literal>GeneraredSmoketest</literal>.</simpara> |
| <section xml:id="how-to-handle-smoke-test-errors"> |
| <title>How to handle smoke test errors?</title> |
| <simpara>If a smoke test fails, the concrete input should be extracted into a dedicated error test case. The existing classes like <literal>scoping.ErrorTest</literal> should be used to add the broken input document and create fast running cases for them. For that purpose, the <literal>ExceptionAnalyzer</literal> can be used. Such a test case will usually look like this:</simpara> |
| <screen>@Test |
| def void testNoNPE_03() { |
| val script = `` ' |
| var target = { |
| s: "hello", |
| set x |
| `` '.parse |
| analyser.analyse(script, "script", "script") |
| }</screen> |
| </section> |
| <section xml:id="smoketester-and-exceptionanalyzer"> |
| <title>SmokeTester and ExceptionAnalyzer</title> |
| <simpara>The components that are used to implemement the smoke tests are the <literal>SmokeTester</literal> and the <literal>ExceptionAnalyzer</literal>. The smoke tester performs the permutation of the characters from the input model whereas the analyzer does the heavy lifting of passing the parsed model to various components such as the type system or the validation. Both utilities can be used to add either new smoke test documents or to check for the robustness of an implementation. It’s espeically useful to use the ExceptionAnalyzer in conjunction with existing test suites like the <literal>LibraryParsingTestSuite</literal> since it can be used instead of the other utilities like the <literal>PositiveAnalyzer</literal> or the <literal>NegativeAnalyzer</literal>.</simpara> |
| </section> |
| </section> |
| <section xml:id="sec:UI_Tests_with_SWTBot" role="language-n4js"> |
| <title>UI Tests with SWTBot</title> |
| <simpara>For testing functionality from the end-user perspective we use UI tests based on SWTBot <link xl:href="http://eclipse.org/swtbot/">http://eclipse.org/swtbot/</link>.</simpara> |
| <simpara>Since UI tests are usually rather fragile, it was decided to keep the number of these tests as low as possible. The main purpose of these tests is to ensure that the most fundamental IDE functionality is working properly, e.g. creating an N4JS project, creating a new N4JS file, running N4JS code in node.js.</simpara> |
| <simpara>The tests have a number of SWTBot dependencies. For details, please refer to the latest target platform definition file.</simpara> |
| <section xml:id="writing-swtbot-tests"> |
| <title>Writing SWTBot Tests</title> |
| <simpara>The implementation is contained in bundle <literal>org.eclipse.swtbot.tests</literal>. Writing SWTBot tests is straightforward; see source code of <literal>AbstractSwtBotTest</literal> for usage and examples.</simpara> |
| <simpara>Some hints:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>Many methods of the SWTBot framework already include assertions. For example, the method <literal># menu(String)</literal> to find a particular menu or menu item will assert that the item was found and otherwise throw an exception. Therefore, it is safe to write code like:</simpara> |
| <screen>bot.menu("File").menu("New").menu("Project...").click();</screen> |
| </listitem> |
| <listitem> |
| <simpara>It is usually considered bad practice to use sleep delays in UI tests. Instead, wait for the element to appear using methods such as <link xl:href="http://download.eclipse.org/technology/swtbot/helios/dev-build/apidocs/org/eclipse/swtbot/swt/finder/SWTBot.html#waitUntil">http://download.eclipse.org/technology/swtbot/helios/dev-build/apidocs/org/eclipse/swtbot/swt/finder/SWTBot.html#waitUntil</link> or <link xl:href="http://download.eclipse.org/technology/swtbot/helios/dev-build/apidocs/org/eclipse/swtbot/swt/finder/SWTBot.html#waitWhile">http://download.eclipse.org/technology/swtbot/helios/dev-build/apidocs/org/eclipse/swtbot/swt/finder/SWTBot.html#waitWhile</link>.</simpara> |
| </listitem> |
| </itemizedlist> |
| </section> |
| <section xml:id="running-swtbot-tests"> |
| <title>Running SWTBot Tests</title> |
| <simpara>To run the tests locally in Eclipse just right-click the bundle and select <literal>Run As > SWTBot Test</literal> . To run them locally via maven simply start a normal maven build, no additional command line arguments, etc. required.</simpara> |
| <simpara>To run remotely in a Jenkins build: on Windows the build must be executed with a logged-in user; on Linux Xvfb and a window manager are required. The recommended window manager is metacity. More information can be found here: <link xl:href="http://wiki.eclipse.org/SWTBot/Automate_test_execution">http://wiki.eclipse.org/SWTBot/Automate_test_execution</link>.</simpara> |
| <simpara>To use metacity, add the following pre-build shell command to the Jenkins build configuration:</simpara> |
| <programlisting language="bash" linenumbering="unnumbered">sleep 5; metacity --replace --sm-disable &</programlisting> |
| <simpara>The sleep is required because metacity depends on Xvfb to be fully initialized, which might take a moment on slower build nodes. The following Jenkins console log shows the expected output when starting Xvfb and metacitiy:</simpara> |
| <programlisting language="bash" linenumbering="unnumbered">Xvfb starting$ Xvfb :1 -screen 0 1024x768x24 -fbdir /var/lib/build/2014-09-05_10-36-343337290231975947044xvfb |
| [N4JS-IDE_Oliver] $ /bin/sh -xe /tmp/hudson4475531813520593700.sh |
| + sleep 5 |
| + metacity --replace --sm-disable |
| Xlib: extension "RANDR" missing on display ":1.0". |
| Window manager warning: 0 stored in GConf key /desktop/gnome/peripherals/mouse/cursor_size is out of range 1 to 128 |
| Process leaked file descriptors. See http://wiki.jenkins-ci.org/display/JENKINS/Spawning+processes+from+build for more information |
| Parsing POMs |
| ...</programlisting> |
| <simpara>The warnings and error messages in the above log are expected and are considered unharmful (cf. discussion with Jörg Baldzer).</simpara> |
| </section> |
| </section> |
| <section xml:id="sec:Debugging_UI_Tests"> |
| <title>Debugging UI Tests</title> |
| <simpara>In rare cases UI Tests behave differently depending on the underlying OS und the power of the test machine. Missing user interaction on the build-server often results in waiting processes which in turn get a timeout request from the driving unit-testing-framework. To investigate the UI state on the build node a X11 connection needs to established.</simpara> |
| <section xml:id="sec:Connecting_to_the_X_server_on_the_build_node"> |
| <title>Connecting to the X-server on the build-node</title> |
| <simpara>First a vnc server needs to be started on the build-node. This is done via <literal>x11vnc -display :1 -shared > /x11vnc.log 2>&1 &</literal> as a pre-build shellscript-step in Jenkins.</simpara> |
| <simpara>You can narrow down the build to run on a specific node in Jenkins if a repeatable environment is required otherwise the current build-node is listed in the overview page. For security reasons the connection needs to be tunneld through an ssh login:</simpara> |
| <programlisting language="bash" linenumbering="unnumbered">ssh -l viewer -L 5900:localhost:5900 build-linux-25 ’x11vnc -localhost -display :1</programlisting> |
| <simpara>Here we are using the mating <literal>build-linux-25</literal> with the generic user/password ’viewer’ to start the program <literal>x11vnc</literal>. For the actual display number – <literal>:1</literal> in this case – you can refer to the console output. The command above tunnels the default vnc port 5900. You can now connect on <literal>localhost</literal> with a vnc client. If the user is not available, the <literal>x11vnc</literal> program not installed or in case of other issues, ask the build-and-release team.</simpara> |
| <simpara>To display the screen you can use any arbitrary vnc-client (on mac there is screen sharing, in theory just opened from the command line by hitting <literal>open vnc://viewer:viewer@localhost:5900</literal>). One working client is ’chicken of the vnc’ <link xl:href="http://sourceforge.net/projects/cotvnc/">http://sourceforge.net/projects/cotvnc/</link></simpara> |
| </section> |
| <section xml:id="sec:Tools_to_investigate_the_java_stack"> |
| <title>Tools to investigate the java-stack</title> |
| <simpara>Unix and Java come with some useful programs for investigations. The following tools may need some advanced rights to see processes from other users.</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><literal>htop</literal> enhanced top to see all currently running processes</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>jps</literal> list running java processes</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>jstack</literal> investigate running java process</simpara> |
| </listitem> |
| </itemizedlist> |
| </section> |
| </section> |
| </chapter> |
| <chapter xml:id="_ui-concepts"> |
| <title>UI Concepts</title> |
| <warning> |
| <simpara>Parts of this chapter may be outdated.</simpara> |
| </warning> |
| <section xml:id="sec:User_Interface_Concepts" role="language-n4js"> |
| <title>User Interface Concepts</title> |
| <section xml:id="sec:Eclipse_UI_Concepts"> |
| <title>Eclipse UI Concepts</title> |
| <simpara>The following list gives an overview of Eclipse specific UI concepts and which classes are used for implementation.</simpara> |
| <section xml:id="sec:Label_Provider"> |
| <title>Label Provider</title> |
| <simpara>Also provides decorations for icons and text labels.</simpara> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Example</simpara> |
| </entry> |
| <entry> |
| <simpara>The representation of objects in the outline view or in search results.</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Eclipse API</simpara> |
| </entry> |
| <entry> |
| <itemizedlist> |
| <listitem> |
| <simpara><literal>o.e.jface.viewers.ILabelProvider</literal> → without styes</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>o.e.jface.viewers.DelegatingStyledCellLabelProvider.IStyledLabelProvider</literal> → with styles Drawback: Depends on Image rather than ImageDescriptor (see below)</simpara> |
| </listitem> |
| </itemizedlist> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Xtext Specifics</simpara> |
| </entry> |
| <entry> |
| <itemizedlist> |
| <listitem> |
| <simpara><literal>Declarative API</literal> via reflective, polymorphic dispatching. <literal>org.eclipse.xtext.ui.label.AbstractLabelProvider</literal></simpara> |
| </listitem> |
| <listitem> |
| <simpara>Allows to work with ImageDescriptors (non-ui-thread, can be composed), but cumbersome</simpara> |
| </listitem> |
| <listitem> |
| <simpara>DefaultLabelProvider will be used everywhere (outline etc.), returns the <literal>name</literal> (via reflection). You could bind specific label providers.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>DescriptionLabelProvider provides labels for objects from the Xtext index used by <literal>open model element</literal> , <literal>find references</literal> – the later is already customized)</simpara> |
| </listitem> |
| </itemizedlist> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Best Practices</simpara> |
| </entry> |
| <entry> |
| <simpara>Labels are often use-case specific, so a single label provider is not always useful Therefore, in Xtext are different label providers for different use cases. Use cases are defined and enumerated (see <literal>DefaultUiModule.configure*Label*</literal>). |
| + |
| Often labels could be easier created where they are needed instead of using a label provider (even for things like OutlineView). (LabelProvider are maybe over-engineered). (Note: default label provider has dependencies to SWT, because it uses images, which are often not needed; also they are only called with an object and no further configuration). |
| + |
| Images: |
| +</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>DeclarativeLabelProvider: text and image (String: path relative to icons folder; or: ImageDescriptor, or Image), put it into folder (needs to be called <literal>icons</literal> , otherwise bind another name)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>better use AbstractLabelProvider and use EMF concepts (easier to debug and handle)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>image format and size: sub-folders with 32, 12, 16, 24, etc. (look which sizes are needed); in about view maybe bigger; png (supports transparency)</simpara> |
| </listitem> |
| </itemizedlist> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section xml:id="sec:Markers"> |
| <title>Markers</title> |
| <blockquote> |
| <attribution> |
| Eclispe Help |
| </attribution> |
| <simpara>Markers are objects that may be associated with Workbench resources. There are many uses of markers in the Workbench…​ |
| Markers are shown in a marker view (Tasks, Problems or Bookmark view) or on the marker bar in the editor area.</simpara> |
| </blockquote> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Examples</simpara> |
| </entry> |
| <entry> |
| <simpara>Tasks, Problems, Bookmarks, Breakpoints, Trace information</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Eclipse API</simpara> |
| </entry> |
| <entry> |
| <simpara><literal>org.eclipse.core.resources.IMarker</literal> and <literal>IResource.findMarkers</literal> |
| +</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>Marker types registered via extension point (basically String). A marker is more or less a Map of String → String with some meta information, e.g. the resource location in WS, line numbers, type</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Markers are very efficient (e.g., find markers of a certain type), cf Xtext Specifics</simpara> |
| </listitem> |
| <listitem> |
| <simpara>For validation, some new marker types are already registered.</simpara> |
| </listitem> |
| </itemizedlist> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Xtext</simpara> |
| </entry> |
| <entry> |
| <simpara>(2.6) |
| +</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>Todo-Markers are created during build, task list is populated by these markers.</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>org.eclipse.xtext.tasks.ITaskFinder</literal> (and default implementation is bound by default, can be replaced with custom implementation)</simpara> |
| </listitem> |
| </itemizedlist> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Best Practices</simpara> |
| </entry> |
| <entry> |
| <simpara>In Xtend, markers are used to trace from original file to generated file. They are hidden (and not displayed), so in general markers can be used for non-UI-problems as well (but only available in Eclipse of course) clean up markers: no general solution, often managed by some (single) life-cycle aware class (e.g., Builder)</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section xml:id="sec:Commands__Toolbar_and_Menus"> |
| <title>Commands, Toolbar and Menus</title> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Examples</simpara> |
| </entry> |
| <entry> |
| <simpara>Organize Imports,</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Eclipse API</simpara> |
| </entry> |
| <entry> |
| <simpara>use Commands + Handlers instead of Actions (or ActionDelegates etc.)</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Xtext Specifics</simpara> |
| </entry> |
| <entry> |
| <simpara>uses commands and handlers</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Best Practice</simpara> |
| </entry> |
| <entry> |
| <simpara><emphasis role="strong">Use commands and handlers</emphasis> - |
| handler usually only delegates to real thing (that is, retrieve parameters from context and call the real thing)<?asciidoc-br?> |
| <emphasis role="strong">Register in pluginxml</emphasis> via ExecutableExtensionFactory to be able to use injection (also pre-generatd, e.g.:</simpara> |
| <programlisting language="xml" linenumbering="unnumbered"> <handler class="org.eclipse.n4js.ui.N4JSExecutableExtensionFactory:org.eclipse.xtext.ui.editor.handler.ValidateActionHandler" |
| commandId="org.eclipse.n4js.N4JS.validate"></programlisting> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| <simpara><emphasis role="strong">Undo</emphasis>: use TextEdit and MultiTextEdit (composed)<?asciidoc-br?> |
| otherwise very low level</simpara> |
| </section> |
| <section xml:id="sec:Content_Assist"> |
| <title>Content Assist</title> |
| <blockquote> |
| <attribution> |
| Eclipse Help |
| </attribution> |
| <simpara>Content assist allows you to provide context sensitive content completion upon user request. Popup windows (infopops) are used to propose possible text choices to complete a phrase. The user can select these choices for insertion in the text. Content assist also supports contextual infopops for providing the user with information that is related to the current position in the document.</simpara> |
| </blockquote> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Risk</simpara> |
| </entry> |
| <entry> |
| <simpara>Always needs longer than anticipated.</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Examples</simpara> |
| </entry> |
| <entry> |
| <simpara>complete name of function in function call, complete keywords</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Eclipse API</simpara> |
| </entry> |
| <entry> |
| <itemizedlist> |
| <listitem> |
| <simpara>IContentAssistant wraps the widget,</simpara> |
| </listitem> |
| <listitem> |
| <simpara>IContentProposalProvider computes the (array of) CompletionProposal (quite cumbersome!).</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Many extension interfaces that provide valuable UI features.</simpara> |
| </listitem> |
| </itemizedlist> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Xtext Specifics</simpara> |
| </entry> |
| <entry> |
| <itemizedlist> |
| <listitem> |
| <simpara>ConfigurableCompletionProposal implements the currently defined extension interfaces, provides getters to modify the proposal after the fact.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Context: The ContentAssistContext is provided by the framework according to the current cursor position in the document (cf. <literal>ContentAssisParser</literal>), semantic context (semantic element) computed with best match strategy (worst case you get the root element). Multiple contexts may be valid at the very same cursor position since the replace region may be different for different proposals.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Various abstracts above the JFace stuff are available in Xtext, some of the <literal>over the top</literal> , others quite handy.</simpara> |
| </listitem> |
| </itemizedlist> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Best Practices</simpara> |
| </entry> |
| <entry> |
| <itemizedlist> |
| <listitem> |
| <simpara>List of follow elements can be supposed to be complete, no need to figure out them with regular expressions etc.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>in rare cases it is necessary to <literal>manually</literal> scan the text context, e.g. to get the variable name based on the variable type. → we will provider a utility class for that using regex. NEVER search on the text with simple string methods.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>In N4JSProposalProvider, override pre-generated methods (see <literal>AbstractN4JSProposalProvider</literal>) – do not overload (with concrete semantic element)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>how to implement complete-methods:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>inspect context, examine current semantic element provide elements from scope or hard coded proposal: see <xref linkend="sec:Proposals"/></simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| </itemizedlist> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section xml:id="sec:Quick_Fixes"> |
| <title>Quick Fixes</title> |
| <blockquote> |
| <attribution> |
| Eclipse Help |
| </attribution> |
| <simpara>Users can select a problem marker and choose a Quick Fix from a popup containing the list of supplied fixes contributed for the marker.</simpara> |
| </blockquote> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Examples</simpara> |
| </entry> |
| <entry> |
| <simpara>Add Import, Add Override Annotation</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Eclipse API</simpara> |
| </entry> |
| <entry> |
| <simpara>Based on ICompletionProposal (powerful) |
| +</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>QuickFixes are registered to marker (marker attribute: is fixable or not – this attribute is a guess only, there does not need to be a quick fix)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>MarkerResolutionGenerator (can also be used to fix several markers at once)</simpara> |
| </listitem> |
| </itemizedlist> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Xtext Specifics</simpara> |
| </entry> |
| <entry> |
| <simpara>Based on ISematicModification (seemingly powerful but in fact weak) and IModification (less weak, but still very weak compared to ICompletionProposal) – only creates DocumentChanges. |
| + |
| Declarativ API that links to issue codes via annotations on 'fix' methods in AbstractDeclarativeQuickfixProvider.</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Best Practices</simpara> |
| </entry> |
| <entry> |
| <simpara>ICompletionProposal vs. DocumentChanges, ICompletionProposal is much more powerful. IModifications can also provide semantic changes, but not really recommended |
| +</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>Associated to isses via IssueCodes, @Fix similar to @Check API – only less powerful Xtext abstraction (no ICompletionProposal)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>use issue data to provide hints for fix labels (which should be fast!) or solution strategies (but only strings) → do not compute the label for the fix from the model!</simpara> |
| </listitem> |
| <listitem> |
| <simpara>share code between checks and fixes → no built-in pattern, come up with utility methods (maybe define conventions)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>maybe Sebastian can add a solution that more information is available via @Fix-approach</simpara> |
| </listitem> |
| <listitem> |
| <simpara>no order of quickfixes (sorted by name and priority, latter is not provided by default)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>there can be several @Fix for a single issue code, or pass arbitrary number of resolution to the acceptor</simpara> |
| </listitem> |
| <listitem> |
| <simpara>for most cases simple Xtext quick fix api is good enough (e.g. all Xtend quick fixes use that) |
| + |
| → Xtext feature request: solve multiple markers at a time (possible to do that right now: bind custom <literal>XtextQuickAssistProcessor</literal>, override <literal>MarkerResolutionGenerator.getAdaptedResolutions(List<IssueResolution></literal>) – return a WorkbenchMarkerResolution)</simpara> |
| </listitem> |
| </itemizedlist> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section xml:id="sec:Quick_Assist"> |
| <title>Quick Assist</title> |
| <blockquote> |
| <simpara>"Quick assists perform local code transformations. They are invoked on a selection or a single cursor in the Java editor and use the same shortcut as quick fixes (Ctrl+1), but quick assist are usually hidden when an error is around. To show them even with errors present on the same line, press Ctrl+1 a second time." (Eclipse Help)</simpara> |
| </blockquote> |
| <simpara><literal>like a quickfix without a problem</literal></simpara> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Examples</simpara> |
| </entry> |
| <entry> |
| <simpara>Add/remove inferred types</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Eclipse API</simpara> |
| </entry> |
| <entry> |
| <simpara>Takes cursor position</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Xtext</simpara> |
| </entry> |
| <entry> |
| <simpara>no Xtext support, e.g. no default implementation (XtextQuickAssistProcessor is a quick fix provider, has nothing to do with QuickAssist) but: XtextQuickAssistProcessor, override canAssist, override computeQuickAssistProposals</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section xml:id="sec:Clean_Up_Actions"> |
| <title>Clean Up Actions</title> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Examples</simpara> |
| </entry> |
| <entry> |
| <simpara>Remove unused local vars, sort members</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Eclipse API</simpara> |
| </entry> |
| <entry> |
| <simpara>None, JDT specific (see ICleanUp)</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Xtext Specifics</simpara> |
| </entry> |
| <entry> |
| <simpara>None</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Best Practice</simpara> |
| </entry> |
| <entry> |
| <simpara>Monkey sees - Monkey does (look at JDT), In the end a it’s a CompositeRefactoring, which is a CompletionProposal</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section xml:id="sec:Save_Actions"> |
| <title>Save Actions</title> |
| <simpara>Similar to clean up actions but performed on save</simpara> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Examples</simpara> |
| </entry> |
| <entry> |
| <simpara>Format on save, Organize imports on save</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Eclipse API</simpara> |
| </entry> |
| <entry> |
| <simpara>None, JDT specific (see IPostSaveListener)</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Xtext Specifics</simpara> |
| </entry> |
| <entry> |
| <simpara>None</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Best Practice</simpara> |
| </entry> |
| <entry> |
| <simpara>XtextDocumentProvider.doSaveDocument (maybe better solutions in the future ;-) )</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section xml:id="sec:Auto_Edit"> |
| <title>Auto Edit</title> |
| <simpara>Auto edit is about closing braces that just have been typed, adding indentation after a line break the code snippet <literal>if (true)</literal> so basically it should be unobtrusive typing aids.</simpara> |
| <simpara>By default, restore model structure when editing (guide the user to proper text formatting, help the parser). Should not be used for other purposes in order to not hinder the user’s flow of editing.</simpara> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Examples</simpara> |
| </entry> |
| <entry> |
| <simpara>( → ( <cursor> )</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Eclipse API</simpara> |
| </entry> |
| <entry> |
| <simpara>org.eclipse.jface.text.IAutoEditStrategy</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Xtext Specifics</simpara> |
| </entry> |
| <entry> |
| <simpara>org.eclipse.xtext.ui.editor.autoedit.AbstractEditStrategy, some utility methods + implements VerifyKeyListener. May use the ISourceViewer via implements ISourceViewerAware</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Best Practices</simpara> |
| </entry> |
| <entry> |
| <simpara>Keep it as it is.</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| <simpara>Fun example but not useful in practice cf. <link xl:href="https://code.google.com/a/eclipselabs.org/p/xtext-forms-integration/source/browse/trunk/plugins/org.eclipse.xtext.example.domainmodel.ui/src/org/eclipse/xtext/example/ui/autoedit/FantasticAutoEditStrategy.java?r=19">FantasticAutoEditStrategy</link></simpara> |
| </section> |
| <section xml:id="sec:Template_Proposals"> |
| <title>Template Proposals</title> |
| <simpara>More sophisticated edit utils that are invoked by means of content assist.</simpara> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Examples</simpara> |
| </entry> |
| <entry> |
| <simpara>sysout → System.out.println(<literal><cursor></literal> );</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Eclipse API</simpara> |
| </entry> |
| <entry> |
| <simpara>Part of the completion proposal API, e.g. ICompletionProposal</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Xtext Specifics</simpara> |
| </entry> |
| <entry> |
| <simpara>org.eclipse.xtext.ui.editor.contentassist.ITemplateProposalProvider, template contexts along the grammar rules by default, need to be stripped down to become usable.</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Best Practice</simpara> |
| </entry> |
| <entry> |
| <simpara>ship some: create them manually in workbench, export them as XML, fix XML file (add IDs, in Xtext documentation), put XML file in folder <literal>templates</literal> in UI plugin where propose a certain proposal: customize XtextTemplateContextTypeRegistry (bind subclass, override register context types) – by default too many context types are registered placeholders inside templates specific to Xtext – RTFM</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section xml:id="sec:Outline_View___Quick_Outline"> |
| <title>Outline View / Quick Outline</title> |
| <simpara>Structural represenation of the file contents (usually with different filter and sorting strategies).</simpara> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Examples</simpara> |
| </entry> |
| <entry> |
| <simpara>Outline View (but not Navigator nor package explorer), Quick Outline (in Xtext: same provider)</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Eclipse API</simpara> |
| </entry> |
| <entry> |
| <simpara>org.eclipse.ui.views.contentoutline.IContentOutlinePage</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Xtext Specifics</simpara> |
| </entry> |
| <entry> |
| <simpara>Lazy tree creation, syncing via EObject ranges, thread save access to the EObject from nodes. Declarative API to create the tree contents. org.eclipse.xtext.ui.editor.outline.impl.DefaultOutlineTreeProvider |
| + |
| allow actions on outline nodes (e.g., goto referenced file in <literal>import</literal> of outline)</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Best Practice</simpara> |
| </entry> |
| <entry> |
| <itemizedlist> |
| <listitem> |
| <simpara>Produced from semantic model, tree structure of outline nodes</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>show tree based on TypeModel, maybe filter out elements w/o SyntaxElements (with type model, this should be rather cheap!)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>use icons and styled labels (first user impression!)</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara>May run in the background (BackgroundOutlineTreeProvider)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>done lazily</simpara> |
| </listitem> |
| <listitem> |
| <simpara>workflow: reconceiler: outline is a model listener</simpara> |
| </listitem> |
| </itemizedlist> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| <simpara>Helpful tools for icons in outline view:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><link xl:href="http://marketplace.eclipse.org/content/eclipse-icon-archive-tool">Eclipse view</link> to show available Eclipse icons (that are of course licenced under EPL) with possibility to export them (<link xl:href="http://bwgz-org.googlecode.com/files/EclipseIconArchiveTool-1.pdf">documentation</link>)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>overview of Eclipse icons: <link xl:href="http://eclipse-icons.i24.cc/">http://eclipse-icons.i24.cc/</link></simpara> |
| </listitem> |
| </itemizedlist> |
| </section> |
| <section xml:id="sec:Navigator__Package_Explorer__Project_Explorer"> |
| <title>Navigator, Package Explorer, Project Explorer</title> |
| <simpara>three <literal>explorers</literal> , Navigator <literal>latest</literal> and most extensible one</simpara> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Best Practices</simpara> |
| </entry> |
| <entry> |
| <simpara>use Navigator only! (RTFM, nothing specific to Xtext yet)</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| <simpara>cf. <link xl:href="http://projects.eclipse.org/projects/technology.handly">http://projects.eclipse.org/projects/technology.handly</link> <literal>read index and show it in the navigator</literal></simpara> |
| </section> |
| <section xml:id="sec:Hyperlinking_and_Navigation"> |
| <title>Hyperlinking and Navigation</title> |
| <simpara>Linking (propose multiple linking targets, e.g. goto declaration or goto implementation when CTRL (or other modifier) + Left Mouse Click on method when receiver type is interface - show all available implementations)</simpara> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Examples</simpara> |
| </entry> |
| <entry> |
| <simpara>Go to declaration, Go to implementation, Go to super</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Eclipse API</simpara> |
| </entry> |
| <entry> |
| <simpara>org.eclipse.jface.text.hyperlink.IHyperlinkDetector</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Xtext Specifics</simpara> |
| </entry> |
| <entry> |
| <simpara>org.eclipse.xtext.ui.editor.hyperlinking.DefaultHyperlinkDetector, navigation to EObject URI most interesting: SIGNIFICANT cf. org.eclipse.xtext.resource.ILocationInFileProviderExtension.RegionDescription</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Best Practice</simpara> |
| </entry> |
| <entry> |
| <itemizedlist> |
| <listitem> |
| <simpara>subclass and bind IHyperlinkHelper (returns an array of possible links, first one is the default)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>also see ILocationInFileProviderExtension (cf. navigation to syntax elements instead of types)</simpara> |
| </listitem> |
| </itemizedlist> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section xml:id="sec:Syntax_and_Semantic_Coloring"> |
| <title>Syntax and Semantic Coloring</title> |
| <simpara>Coloring based on the lexical tokens or based on the semantic tokens (the parsed model). The parser may treat certain lexical keywords as valid identifiers in some contexts. Some of those should not appear as keywords. Semantic coloring is usually more expensive to compute thus run in the background and with some delay</simpara> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Examples</simpara> |
| </entry> |
| <entry> |
| <simpara>Numbers, String literals (lexical) Escape sequences in Strings, method calls, property read / write access (semantic)</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Eclipse API</simpara> |
| </entry> |
| <entry> |
| <simpara><literal>org.eclipse.jface.text.presentation.IPresentationDamager</literal> <literal>org.eclipse.jface.text.presentation.IPresentationRepairer</literal> <literal>org.eclipse.jface.text.rules.ITokenScanner</literal> |
| + |
| Scan for tokens and associate text attributes with tokens. Compute the region of the document that has to be recolored after a text change. Tokens may not overlap. |
| + |
| Also Eclipse provides Themes that are styled via CSS. Coloring can be adjusted to themes where the logical names are mapped to different default values.</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Xtext Specifics</simpara> |
| </entry> |
| <entry> |
| <itemizedlist> |
| <listitem> |
| <simpara><literal>o.e.x.ui.editor.syntaxcoloring.ITextAttributeProvider</literal> - associate Antlr token names with coloring styles (logical names of text coloring)</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>o.e.x.ui.editor.syntaxcoloring.AbstractAntlrTokenToAttributeIdMapper</literal>- convert the antlr tokens to JFace ITokens with proper text applied</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>o.e.x.ui.editor.syntaxcoloring.IHighlightingConfiguration</literal> - register logical text colorings with default values, yields a preference page and the proper configuration for the text attribute provider</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>o.e.x.ui.editor.syntaxcoloring.ISemanticHighlightingCalculator</literal> - traverse the AST and associate arbitrary ranges of the text with.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>logical coloring names (this is a key to a style stored in the preference store), if multiple styles are returned, styles will be merged if they overlap (and if possible); JFace constraints are implicitly fulfilled</simpara> |
| </listitem> |
| </itemizedlist> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Best Practice</simpara> |
| </entry> |
| <entry> |
| <itemizedlist> |
| <listitem> |
| <simpara>subclass DefaultSemanticHighlightingCalculator and bind ISemanticHighlightingCalculator</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>traverse resource from left to right (usually order of semantic elements – small performance improvement)</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara>provide new logical style: subclass DefaultHighlightingConfiguration and bind IHighlightingConfiguration; override configure (see overridden)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>semantic coloring always wins</simpara> |
| </listitem> |
| <listitem> |
| <simpara>only a few decisions can me made in lexical coloring, override lexical:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>subclass <literal>DefaultAntlrTokenToAttributeIdMapper</literal> bind <literal>TokenTypeToStringMapper</literal></simpara> |
| </listitem> |
| <listitem> |
| <simpara>e.g., color jsdoc comments differently to multiline, regex</simpara> |
| </listitem> |
| <listitem> |
| <simpara>e.g. color tags inside jsdocs or regex inside, use semantic coloring</simpara> |
| </listitem> |
| <listitem> |
| <simpara>lexical: different kind of keywords (e.g., N4JS keywords vs. JS keywords)</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara>change coloring (via toggle button), possible approach:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>(inject singleton into highlighter, state of singleton is changed by toggle button, listen to that object in the editor, calculator cannot be triggered from outside due to UI-thread issues)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>prefered: store state in preference store and get the information then from there in the hightligher, inject PreferencestoreAccess in Calculator</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| </itemizedlist> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section xml:id="sec:Code_Formatter"> |
| <title>Code Formatter</title> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Examples</simpara> |
| </entry> |
| <entry> |
| <simpara>Auto-Format Source Code, Auto-Format code inserted by code-rewrite</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Eclipse API</simpara> |
| </entry> |
| <entry> |
| <simpara><literal>org.eclipse.jface.text.formatter.IContentFormatter</literal> - here is the document and some range - modify at will</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Xtext Specifics</simpara> |
| </entry> |
| <entry> |
| <simpara>Declarative Formatting API (to be deprecated) - associate formatting rules with grammar elements New formatting API (mixture of declarative and imperative) - here is the model, do what you want (space before, linebreak after, indentation increase / decrease), the engine will merge your advices and apply them to the document</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Best Practice</simpara> |
| </entry> |
| <entry> |
| <simpara>wait for 2.8 (maybe in 2.7.x)</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section xml:id="sec:Wizards"> |
| <title>Wizards</title> |
| <blockquote> |
| <attribution> |
| Eclipse Help |
| </attribution> |
| <simpara>Wizards are used to guide the user through a sequenced set of tasks. Your plug-in can contribute wizards at predefined extension points in the workbench. It can also create and launch its own wizards.</simpara> |
| </blockquote> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Examples</simpara> |
| </entry> |
| <entry> |
| <simpara>New N4JS Class</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Eclipse API</simpara> |
| <simpara>Xtext Specifics</simpara> |
| </entry> |
| <entry> |
| <itemizedlist> |
| <listitem> |
| <simpara>Xtend based Wizards</simpara> |
| </listitem> |
| <listitem> |
| <simpara>also see Formular Editor for Embedded Xtext editor</simpara> |
| </listitem> |
| </itemizedlist> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Best Practices</simpara> |
| </entry> |
| <entry> |
| <itemizedlist> |
| <listitem> |
| <simpara>use preferences (could be hidden, so use them even if not made configurable to the user)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>use standard JFace wizard API, use Xtend template expressions for file templates</simpara> |
| </listitem> |
| </itemizedlist> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section xml:id="sec:Cheat_Sheets"> |
| <title>Cheat Sheets</title> |
| <blockquote> |
| <attribution> |
| Eclipse Help |
| </attribution> |
| <simpara>Composite cheat sheets provide guidance through complex problems by breaking the problem into a set of smaller tasks. Composite cheat sheets are registered using the the <literal>org.eclipse.ui.cheatsheets.cheatSheetContent</literal> extension point.</simpara> |
| </blockquote> |
| <simpara>(In Scala IDE: Work Sheets), often combined with Code Snippets</simpara> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Examples</simpara> |
| </entry> |
| <entry> |
| <simpara>Create Hello World Application</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Eclipse API</simpara> |
| <simpara>Xtext Specifics</simpara> |
| </entry> |
| <entry> |
| <simpara>None, probably the embedded editor could be used in a REPL (Read-Evaluate-Print-Loop)</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section xml:id="sec:Context_sensitive_Help"> |
| <title>Context-sensitive Help</title> |
| <blockquote> |
| <attribution> |
| Eclipse Help |
| </attribution> |
| <simpara>A focused set of help topics that is related to the current context can be shown to users on demand using context-sensitive help. This form of user assistance is delivered to users when a platform-specific trigger is activated (e.g. F1 key on Windows, Ctrl+F1 on GTK, Help key on Carbon). Until Eclipse 3.1, context-sensitive help was presented in infopop windows. Since 3.1, a new Help view is the preferred way to deliver context-sensitive information to the user.</simpara> |
| </blockquote> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Examples</simpara> |
| </entry> |
| <entry> |
| <simpara>Help in Formular Editor, Help about syntax construct, API-Help</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Eclipse API</simpara> |
| <simpara>Xtext Specifics</simpara> |
| </entry> |
| <entry> |
| <simpara>None</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section xml:id="sec:Hovers"> |
| <title>Hovers</title> |
| <simpara>Hover allow to display additional information as soon as the cursor stays on a certain text region. Some hovers can be requested by shortcuts (e.g. F2) similar to sort of an online help.</simpara> |
| <simpara>Different kind of hovers may appear depending on the context, e.g. the error hover will have higher prio than the documentation hover. Different modifier keys may be assigned to request different hover kinds while hovering a region with the mouse. (didn’t a proper code pointer, though)</simpara> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Examples</simpara> |
| </entry> |
| <entry> |
| <simpara>Hover over method shows JSDoc, Signatures or inferred types, error / problem details</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Eclipse API</simpara> |
| </entry> |
| <entry> |
| <simpara><literal>org.eclipse.jface.text.ITextHover</literal> + <literal>ITextHoverExtension*</literal> - compute hover based on the region that is hovered. Various indirections with <literal>IInformationControl</literal> and <literal>IInformationControlCreator</literal> with many extension interfaces</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Xtext Specifics</simpara> |
| </entry> |
| <entry> |
| <simpara><literal>org.eclipse.xtext.ui.editor.hover.IEObjectHover</literal> - compute hover based on <literal>EObjects</literal></simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Best Practice</simpara> |
| </entry> |
| <entry> |
| <simpara>see XBase hover stuff</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section xml:id="sec:Folding"> |
| <title>Folding</title> |
| <simpara>Code folding allows to skip parts of the code that are mandatory semantically but usually do not provide added value for the reader, e.g. import sections</simpara> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Examples</simpara> |
| </entry> |
| <entry> |
| <simpara>Import section folding, folding of arbitrary methods or comments</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Eclipse API</simpara> |
| </entry> |
| <entry> |
| <simpara>Not much there, most of that stuff is implemented specific to JDT or ODE. Projections usually only work per line, that is, a subsection of a line cannot be folded, e.g. it’s not possible to show</simpara> |
| <screen>var x = new Map<String, List<Pair<String, Number>>>()</screen> |
| <simpara>as</simpara> |
| <screen>var x = new Map<...>()</screen> |
| <simpara>Line only limitation in SWT (a guess, didn’t work for Sebastian otherwise)</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Xtext Specifics</simpara> |
| </entry> |
| <entry> |
| <itemizedlist> |
| <listitem> |
| <simpara><literal>org.eclipse.xtext.ui.editor.folding.DefaultFoldingRegionProvider</literal> - here is the resource, compute the folding</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>org.eclipse.xtext.ui.editor.folding.DefaultFoldingStructureProvider</literal> - bridge between editor and custom computation, preferences etc would be read from here</simpara> |
| </listitem> |
| <listitem> |
| <simpara>no preference page for folding provided by Xtext</simpara> |
| </listitem> |
| </itemizedlist> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Best Practice</simpara> |
| </entry> |
| <entry> |
| <itemizedlist> |
| <listitem> |
| <simpara>maybe limit to blocks (subclass default, bind to interface)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>probably provide your own folding preference page</simpara> |
| </listitem> |
| </itemizedlist> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section xml:id="sec:Customizable_validation___severity"> |
| <title>Customizable validation / severity</title> |
| <simpara>Some problems are more important to the user than others so they want to change the severity.</simpara> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Examples</simpara> |
| </entry> |
| <entry> |
| <simpara>Deprecation could be an error, warning or ignored (e.g. in test projects)</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Eclipse API</simpara> |
| </entry> |
| <entry> |
| <simpara>None</simpara> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Xtext Specifics</simpara> |
| </entry> |
| <entry> |
| <simpara>IssueSeverityProvider (since 2.6), Monkey sees monkey does: see subclasses of IssueSeverityProvider (we already do that)</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section xml:id="sec:Proposals"> |
| <title>Proposals</title> |
| <simpara>Created by Content Assist, Quick Fixes, Quick Assist.</simpara> |
| <simpara>Basics</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>simplest case: proposals are strings to be inserted</simpara> |
| </listitem> |
| <listitem> |
| <simpara>or displayed string is different from inserted one (e.g. FQN vs. simple)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>ConfigurableCompletionProposal created via factory methods in AbstractN4JSProposalProvider (*create*Pro)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>PrefixMatcher (by default CamelCase aware) – for filtering, it usually is not necessary to use it when computing the proposal (only if it expensive to compute proposals) – that is, prefix can be ignored when computing a proposal because the prefix matcher will filter out invalid proposals anyway</simpara> |
| </listitem> |
| <listitem> |
| <simpara>pass a filter (Guava preodicate) to filter out (semantically invalid) proposals, cf. lookupCrossReference(..) – for the things where there are proposals created by default</simpara> |
| </listitem> |
| <listitem> |
| <simpara>priority defined by an int – for sorting. Cf. ContentProposalPriorities → define default priorities (constant values) in N4JS, do not add some random integers!</simpara> |
| </listitem> |
| <listitem> |
| <simpara>modes: bind RepeatedContentAssistProcessor and enable modeaware in ProposalProvider (e.g. for private members which require a quickfix)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>what could be done in the background: hover, lazy (not prepared) proposals (cf. JDT), Xtext 2.7.; different situations are processed in parallel</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>Several changes (e.g. automatic import):</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><literal>ConfigurableCompletionProposal.setTextApplier</literal></simpara> |
| </listitem> |
| <listitem> |
| <simpara>TextApplier: can open dialogs etc., TextApplier is the callback</simpara> |
| </listitem> |
| <listitem> |
| <simpara>usual case: add text at cursor position and somewhere else:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>get document in TextApplier</simpara> |
| </listitem> |
| <listitem> |
| <simpara>for performance, but also: do not use semantic changes in content assist, because model is broken (you will get funny things) – use model (AST) to get offset, but then insert line breaks etc. → maybe create utility class for retrieving current formattings which are then used in the text edit → maybe provide tools for retrieving certain locations (e.g. import section, field section, etc.)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>do not create model (AST) fragments (which are then serialized), instead directly provide text</simpara> |
| </listitem> |
| <listitem> |
| <simpara>use TextEdit and MultiTextEdit</simpara> |
| </listitem> |
| <listitem> |
| <simpara>set TextViewer redraw to false and to true after the text edits were applied</simpara> |
| </listitem> |
| <listitem> |
| <simpara>have proper TESTS to ensure that file is not broken after the changes</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara><emphasis role="strong">LinkedEditing</emphasis>:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>Linked-Editing mode in ConfigurableCompletionProposal with one editing group only (basically: move the cursor somewhere after editing it, see setSimpleLinkedMode)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>do it manually: cf. LinkedPositionGroup (see call hierarchy of constructor) – used for quick fixes or refactorings rather for content assist</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| </itemizedlist> |
| </section> |
| </section> |
| <section xml:id="sec:Non_Eclipse_UI_Concepts"> |
| <title>Non-Eclipse UI Concepts</title> |
| <simpara>The following entries are not necessarily implemented yet.</simpara> |
| <section xml:id="sec:Overlays"> |
| <title>Overlays</title> |
| <simpara>An overlay is a small annotation similar to an hover, attached to a specific location in the editor and is moved with that location.</simpara> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Examples</simpara> |
| </entry> |
| <entry> |
| <simpara>Show inferred types</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section xml:id="sec:Goto__Inferred__Type"> |
| <title>Goto (Inferred) Type</title> |
| <simpara>Navigate to an inferred type (or other <literal>invisible</literal> information)</simpara> |
| </section> |
| <section xml:id="sec:Postfix_Completion"> |
| <title>Postfix Completion</title> |
| <simpara>(IntelliJ) Replace code <emphasis>AFTER</emphasis> an expression</simpara> |
| </section> |
| </section> |
| </section> |
| <section xml:id="_user-interface-resources"> |
| <title>User Interface Resources</title> |
| <section xml:id="_icons"> |
| <title>Icons</title> |
| <simpara>In parts, the N4JS IDE re-uses some of the icons that come with the Eclipse Platform as well as the Eclipse JDT development environment. However, in some cases we also provide our own icons to illustrate N4JS-specific concepts.</simpara> |
| <section xml:id="_eclipse-platform-icons"> |
| <title>Eclipse Platform Icons</title> |
| <simpara>When re-using the Eclipse Platform Icons, the icons are usually copied over to the <literal>icons/</literal> folder of the <literal>org.eclipse.n4js.ui</literal> bundle. In this folder, the <literal>README.adoc</literal> file keeps book on the origin of all the collected icons (e.g. different Eclipse Projects).</simpara> |
| </section> |
| <section xml:id="_n4js-specific-icons"> |
| <title>N4JS Specific Icons</title> |
| <simpara>In some cases, the icons the Eclipse eco-system provides do not suffice to sensibly express N4JS concepts. In these cases we provide our own icons. When designing those we try to imitate the general Eclipse artstyle in order for our icons to integrate well with the overall appearance of Eclipse.</simpara> |
| <simpara>For the creation of new icons, the <literal>eclipse-svg-icons</literal> repository (see <link xl:href="https://github.com/Seung-Yoon/eclipse-svg-icons">https://github.com/Seung-Yoon/eclipse-svg-icons</link>) has proven helpful. The repository contains raw SVG files which can be used to reproduce the bitmap icons that are contained in, for instance, the <literal>org.eclipse.platform.ui</literal> or <literal>org.eclipse.jdt.ui</literal> bundle. Based on that, common vector-graphics editing software may be used to further adapt color, form and style of existing icons (e.g. Inkscape <link xl:href="https://inkscape.org/en/">https://inkscape.org/en/</link>).</simpara> |
| </section> |
| <section xml:id="_high-resolution-icons"> |
| <title>High Resolution Icons</title> |
| <simpara>With the Neon release, Eclipse SWT introduced explicit support for high-DPI monitors (see <link xl:href="https://www.eclipse.org/eclipse/news/4.6/platform.php#swt-autoscale">https://www.eclipse.org/eclipse/news/4.6/platform.php#swt-autoscale</link>). In order to provide a good user experience, we want to provide high-DPI support for as many of our icons as possible. For that, it suffices to simply provide an alternative resource with higher resolution by appending the prefix @2x to its name (e.g. <literal>class.png</literal> and <literal>class@2x.png</literal>). Code-wise, no adjustments are required. In case of copied Eclipse Platform Icons, most of the time a corresponding 2x-version can be obtained from the original source. In case of N4JS Specific Icons, we export all icons in the resolutions 16x16 and 32x32. For that, it is of particular importance to make sure that the scaling is done in accordance with the native resolution (cf. pixel perfect scaling, also see <link xl:href="https://en.wikipedia.org/wiki/Native_resolution">https://en.wikipedia.org/wiki/Native_resolution</link>).</simpara> |
| </section> |
| </section> |
| </section> |
| </chapter> |
| <chapter xml:id="_formatting"> |
| <title>Formatting</title> |
| <warning> |
| <simpara>This chapter may be outdated.</simpara> |
| </warning> |
| <section xml:id="sec:FmtObjective" role="language-n4js"> |
| <title>Objective</title> |
| <simpara>Writing textual code has many degrees of freedom. The resulting layout differs between authors. Carefully placing whitespace and newlines can increase the readability. Some handling of whitespace can be automated. This chapter describes the techniques used to automate the formatting to some degree.</simpara> |
| <simpara>Formatting N4js source code ensures a consistent style. It takes care of:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>surrounding language constructs with white space to improve readability</simpara> |
| </listitem> |
| <listitem> |
| <simpara>indenting logically grouped elements</simpara> |
| </listitem> |
| <listitem> |
| <simpara>wrapping long lines of code and comments</simpara> |
| </listitem> |
| <listitem> |
| <simpara>inserting semicolons, which would otherwise be automatically inserted (ASI)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>formatting documentation</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>Formatting will never alter the semantics of the code and it will not reorganize it.</simpara> |
| <section xml:id="sec:FmtFormatting_Comments"> |
| <title>Formatting Comments</title> |
| <simpara>N4js distinguishes five different types of comments <emphasis>single line comments</emphasis>, <emphasis>not-indented single line comments</emphasis>, <emphasis>multiline comments</emphasis>, <emphasis>fixed multiline comments</emphasis> and <emphasis>Jsdoc style multiline comments.</emphasis></simpara> |
| <simpara>Single line comments start with <literal>//</literal> and include all characters until the end of line. They usually will be indented and wrapped if they exceed the maximum line length unless they start immediately at the first column. Single line comments starting at position 0 are called <emphasis>not-indented single line comments</emphasis>.</simpara> |
| <simpara>Multiline comments start with <literal>/*</literal> and span all character including line breaks up the the end given by <literal>*/</literal> . The three variants are distinguished by the third and fourth character:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>Comments starting with <literal>/*-</literal> are always fixed multiline comments.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Comments starting with <literal>/**</literal> and following any other character but <literal>*</literal> are Jsdoc style multiline comments. (E.g. <literal>/**+</literal> start Jsdoc but <literal>/***</literal> does not.)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>All others starting with <literal>/*</literal> are ordinary multiline comments.</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara><emphasis>Not-indented single line comments</emphasis> and <emphasis>fixed multiline comments</emphasis> will always remain as they are. Usually they are used to comment out code sections.</simpara> |
| </section> |
| </section> |
| <section xml:id="sec:FmtArchitecture" role="language-n4js"> |
| <title>Architecture</title> |
| <simpara>Formatting mainly takes place in polymorphic dispatch methods <literal>format</literal> in class <literal>N4JSFormatter</literal>. Some language features are formatted with an entry-point using a super-class. This is mainly the case for structures that are sufficiently similar. E.g. <literal>format( FunctionOrFieldAccessor )</literal> is responsible for getter, setter, methods, functions, ….</simpara> |
| <simpara>Some common source-code formattings are grouped by <literal>configureXY()</literal> methods. They get called from the <literal>format</literal> methods for similar code structures, c.f. <literal>configureAnnotations</literal></simpara> |
| <simpara>Since we do not support formatting of the TypeExpression-language stand-alone, this class provides only one format-method throwing an <literal>UnsupportedOperationException</literal> type expressions formatting is defined in class <literal>N4JSFormatter</literal> at the end of the file.</simpara> |
| <figure xml:id="fig:formatter_overview" role="center"> |
| <title>Overview of classes used for formatting</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/17_formatting/images/FormatterArchitecture.svg"/> |
| </imageobject> |
| <textobject><phrase>FormatterArchitecture</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara>The first entry-point for a script is <literal>N4JSFormatter.format(Script, IFormattableDocument)</literal>. Within this method an instance of <literal>N4JSGenericFormatter</literal> is created and used to configure general aspects of automatic-semicolon insertion (put a ’<literal>;</literal>’ where ASI took place in the parser) and handling of colons (’<literal>:</literal>’).</simpara> |
| <section xml:id="sec:Implementation_example"> |
| <title>Implementation example</title> |
| <simpara>Considering the following N4js-snippet where the return value of a function call will be casted to some type.</simpara> |
| <screen>functionCall("a","b") as MyType<string></screen> |
| <simpara>The whole line is a CastExpression comprising of an expression (<literal>functionCall(a,b)</literal>) and a type reference (<literal>MyType<string></literal>).</simpara> |
| <simpara>The <literal>format</literal>-dispatch method written in Xtend would look like:</simpara> |
| <screen> def dispatch void format(CastExpression expr, extension IFormattableDocument document) { |
| expr.regionFor.keyword("as").prepend[newLines = 0; oneSpace].append[newLines = 0; oneSpace]; |
| expr.expression.format; |
| expr.targetTypeRef.format; |
| }</screen> |
| <simpara>In line 2 the format around the keyword <literal>as</literal> is specified where in line 3 and 4 the formatting of the containing elements will be dispatched.</simpara> |
| <simpara>Note that <literal>regionFor</literal> in line 2 is a method declared in <literal>IFormattableDocument</literal> and used via the extension-parameter <literal>document</literal>. It returns an object of type <literal>ISemanticRegionFinder</literal>. Invoking the <literal>keyword</literal> method on this object returns an instance of <literal>ISemanticRegion</literal> which will be passed to the extension methods <literal>prepend</literal> and <literal>append</literal> following the builder pattern. Both <literal>prepend</literal> and <literal>append</literal> take a lambda expression operating on a single parameter of type <literal>IHiddenRegionFormatter</literal>. Inside the lambda-expression this parameter is implicitly used to invoke the methods <literal>setNewLines(0)</literal> and <literal>oneSpace()</literal>. These calls simply disallow line-breaks around as and force the whitespace to be just a single character.</simpara> |
| <simpara>Possible other formatting instructions can be found in <literal>IHiddenRegionFormatter</literal>.</simpara> |
| <simpara>Due to some bugs in auto-wrapping<footnote><simpara>version at the time of writing is Xtext 2.12</simpara></footnote> unsuccessful attempts to wrap a line can insert unexpected new-lines in regions several lines in front of the currently treated source-line.</simpara> |
| <simpara>Debugging the formatter can be cumbersome as, due to GH-12<footnote><simpara><link xl:href="https://github.com/eclipse/xtext-core/issues/12">https://github.com/eclipse/xtext-core/issues/12</link></simpara></footnote>, the <literal>toString()</literal> methods if internal data-structures throw exceptions.</simpara> |
| </section> |
| </section> |
| <section xml:id="sec:FmtFormatter_Implementation_Guidelines"> |
| <title>Formatter Implementation Guidelines</title> |
| <itemizedlist> |
| <listitem> |
| <simpara>Each formatted element should only format it’s inner content. This avoids conflicting situations.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>For each region possibly containing whitespace must be formatted.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Use priorities for conflict-resolutions sparse. For contradicting informations in the same region, the higher priority wins. If both information have the same priority, then an Exception will be thrown, showing two stack-traces to indicate the two code-regions being responsible for the situation.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>For auto-wrapping a callback can be registered. In case of wrapping you can then conditionally change the format. Registering a callback implicitly sets the auto-wrap flag for the region.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Cover formatting with at least two different unit-tests. One having as little white-space as possible (all in one line) and the other as much white-space as possible in order to identify unformatted regions.</simpara> |
| </listitem> |
| </itemizedlist> |
| </section> |
| <section xml:id="sec:FmtConfiguration"> |
| <title>Configuration</title> |
| <simpara>Some formattings can be customised through preference key-value pairs. Class <literal>N4JSFormatterPreferenceKeys</literal> acts as the entry-point to define such key-(default-)value pairs. Some preferences are inherited and are based on values stored in the default preference store (Line length, default tab width, …).</simpara> |
| <simpara>Currently the preferences are not yet accessible by the end-user.</simpara> |
| </section> |
| <section xml:id="sec:FmtUI_Integration"> |
| <title>UI Integration</title> |
| <simpara>Code formatting is invoked with standard key-strokes ( <keycap>CMD</keycap>+<keycap>Shift</keycap>+<keycap>F</keycap> on Mac, <keycap>Ctrl</keycap>+<keycap>Shift</keycap>+<keycap>F</keycap> on Windows)</simpara> |
| <simpara>There is no UI for preferences values yet.</simpara> |
| </section> |
| <section xml:id="sec:FmtUnit_Testing_with_Xpect" role="language-n4js"> |
| <title>Unit Testing with Xpect</title> |
| <simpara>With Xpect Method <literal>formattedLines</literal> implemented in class <literal>org.eclipse.n4js.xpect.FormatterXpectMethod</literal> in bundle <literal>org.eclipse.n4js.tests.helper</literal> the formatting can be tested. The test method requires the number of lines which should be formatted. The desired test is given as a standard multiline expectation.</simpara> |
| <screen>/* XPECT formattedLines 1 --- |
| var a, b, c, d, e; |
| --- */ |
| var a,b,c,d,e;</screen> |
| <simpara>Preferences can be configured in the Xpect setup section by providing string values. Numbers and booleans are converted automatically by the preferences framework.</simpara> |
| <screen>/* XPECT_SETUP org.eclipse.n4js.tests.N4JSXpectTest |
| ResourceSet { |
| ThisFile {} |
| File "wishesImported.n4js" { } |
| } |
| Preference "indentation" " " {} |
| Preference "line.width.max" "100" {} |
| Preference "format.auto_wrap_in_front_of_logical_operator" "false" {} |
| |
| END_SETUP |
| */</screen> |
| <simpara>Tip: Full coverage of the formatting can be tested via authoring the input using spaces as indentation characters if the formatter would use tabs or vice versa. That way untouched lines are distinguishable during the test-runs</simpara> |
| </section> |
| </chapter> |
| <chapter xml:id="_external-libraries"> |
| <title>External Libraries</title> |
| <simpara></simpara> |
| <sidebar> |
| <simpara><link xl:href="https://github.com/eclipse/n4js/issues/1018"><inlinemediaobject> |
| <imageobject> |
| <imagedata fileref="images/issue.svg"/> |
| </imageobject> |
| <textobject><phrase></phrase></textobject> |
| </inlinemediaobject> #1018</link> |
| <link xl:href="https://github.com/eclipse/n4js/issues/397"><inlinemediaobject> |
| <imageobject> |
| <imagedata fileref="images/issue.svg"/> |
| </imageobject> |
| <textobject><phrase></phrase></textobject> |
| </inlinemediaobject> #397</link> |
| <link xl:href="https://github.com/eclipse/n4js/issues/809"><inlinemediaobject> |
| <imageobject> |
| <imagedata fileref="images/issue.svg"/> |
| </imageobject> |
| <textobject><phrase></phrase></textobject> |
| </inlinemediaobject> #809</link> |
| <link xl:href="https://github.com/eclipse/n4js/issues/714"><inlinemediaobject> |
| <imageobject> |
| <imagedata fileref="images/issue.svg"/> |
| </imageobject> |
| <textobject><phrase></phrase></textobject> |
| </inlinemediaobject> #714</link> |
| <link xl:href="https://github.com/eclipse/n4js/issues/653"><inlinemediaobject> |
| <imageobject> |
| <imagedata fileref="images/issue.svg"/> |
| </imageobject> |
| <textobject><phrase></phrase></textobject> |
| </inlinemediaobject> #653</link> |
| <link xl:href="https://github.com/eclipse/n4js/issues/862"><inlinemediaobject> |
| <imageobject> |
| <imagedata fileref="images/issue.svg"/> |
| </imageobject> |
| <textobject><phrase></phrase></textobject> |
| </inlinemediaobject> #862</link> |
| <link xl:href="https://github.com/eclipse/n4js/issues/1133"><inlinemediaobject> |
| <imageobject> |
| <imagedata fileref="images/issue.svg"/> |
| </imageobject> |
| <textobject><phrase></phrase></textobject> |
| </inlinemediaobject> #1133</link></simpara> |
| </sidebar> |
| <simpara><emphasis role="strong">External libraries</emphasis> are N4JS projects that are provided by the N4JS IDE: |
| the <emphasis>built-in</emphasis>/<emphasis>shipped</emphasis> libraries, and all <emphasis>3rd-party libraries</emphasis> that were installed by the <emphasis>N4JS library manager</emphasis>. |
| Each external library consist of a valid package.json file located in the project root and an arbitrary number of files supported by N4JS projects, e.g. <emphasis>.n4js</emphasis>, <emphasis>.njsd</emphasis> and <emphasis>.js</emphasis> files. |
| The purpose of the external libraries is to share and to provide core and third party functionality for N4JS developers both in compile and runtime without rebuilding them.</simpara> |
| <simpara><xref linkend="sec:Built-in_External_Libraries"/> are external libraries that provide some basic functionality for N4JS programmers, such as the class <literal>N4Injector</literal>.</simpara> |
| <simpara><emphasis role="strong">3rd-party libraries</emphasis> are external libraries that are not built-in/shipped with the N4JS IDE. |
| Instead, they can be installed later by the user from third party providers. |
| Currently, only <emphasis>npm packages</emphasis> are supported.</simpara> |
| <simpara>The <emphasis role="strong">N4JS index</emphasis> is populated when the external libraries are compiled. |
| However, this compilation is only triggered through the library manager, but not when building workspace projects. (Self-evidently, the index is also populated when compiling workspace projects.)</simpara> |
| <simpara><emphasis role="strong">Name clashes</emphasis> of projects can happen and they are solved in the following order:</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>User workspace projects always shadow external libraries.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>In case of a name clash between a shipped and a 3rd-party library, the 3rd-party library shadows the shipped project.</simpara> |
| </listitem> |
| </orderedlist> |
| <simpara>The <emphasis role="strong">N4JS library manager</emphasis> is a tool in the N4JS IDE to view and manage external libraries. |
| In particular, the user can (un-)install new 3rd-party libraries, or can trigger the build of all external libraries to re-populate the N4JS index. |
| The library manager also supports other maintenance actions such as deleting all 3rd-party libraries.</simpara> |
| <section xml:id="sec:Major_Components" role="language-n4js"> |
| <title>Major Components</title> |
| <simpara>External libraries are supported based on different components all over the application.</simpara> |
| <simpara>The followings are the most important ones:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><emphasis role="strong">External Resources</emphasis> (<literal>IExternalResource</literal>)</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>These are customized <literal>IResource</literal> implementations for external projects, folders and files.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>With this approach the <literal>IProject</literal>, <literal>IFolder</literal> and <literal>IFile</literal> interfaces have been implemented. Each implementation is backed by a pure <literal>java.io.File</literal> based resource.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>When accessing such external resources for example visiting purposes, getting the members of the resource or simply deleting the resource, internally each requests will be directly performed on the wrapped <literal>java.io.File</literal> without accessing the <literal>org.eclipse.core.resources.IWorkspace</literal> instance.</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara><emphasis role="strong">External Library Workspace</emphasis></simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>This is a kind of dedicated workspace for external libraries and their dependencies.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Any query requests to retrieve a particular project or any dependencies of a particular project via the <literal>IN4JSCore</literal> singleton service will delegated to its wrapped <literal>N4JSModel</literal> singleton. Internally the <literal>N4JSModel</literal> has a reference to a workspace for all the ordinary workspace projects and another reference to the workspace for external libraries. Each query requests will be forwarded to the workspace for the ordinary projects first, and then to the external library workspace. If ordinary project workspace can provide any meaningful response for a request, then the external library workspace will not be accessed at all. Otherwise the query will be executed against the external library workspace. This fallback mechanism provides a pragmatic solution to the project shadowing feature. The project shadowing will be described in details later in this section.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>The <emphasis role="strong">External Library Workspace</emphasis> is only supported and available in the IDE case, in the headless case there are no external libraries available from this dedicated workspace. Since the Xtext index creation and the entire build infrastructure is different, it is supported via target platform file. This is described in more details in a later section (<link linkend="sec:Headless_External_Library_Support">Headless External Library Support</link>]).</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara><emphasis role="strong">External Library Preference Store</emphasis></simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>This preference store is being used to register and un-register external library root folders into its underlying ordered list. A folder is called as an external library root folder if it is neither equal with the Eclipse workspace root nor being nested in the workspace root and contains zero to any external libraries.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Whenever any modifications are being saved in this preference store the <emphasis>External Library Workspace</emphasis> will be updated as well, new libraries will be registered into the workspace and removed libraries will be cleaned up from the workspace.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>When the N4JS IDE application is started in production mode, the initial state of the preference store is being pre-populated with default values. This is necessary to provide built-in libraries to end users. These default values and additional advanced configurations will be mentioned in more details later in this section.</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara><emphasis role="strong">Library Manager</emphasis></simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>This service is responsible for downloading and installing third party <emphasis>npm</emphasis> packages into the <literal>node_modules</literal> folder of the N4JS IDE. After downloading, the newly-installed and/or updated packages are registered as external libraries into the system.</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara><emphasis role="strong">External Library Builder</emphasis></simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>This service is responsible for updating the persistent Xtext index with the currently available external libraries.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Unlike in case of any other ordinary projects, this builder does not triggers a build via the <literal>org.eclipse.core.internal.events.BuildManager</literal> but modifies the persisted Xtext index (<literal>IBuilderState</literal>) directly.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Considers shadowed external libraries when updating the persisted Xtext index.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Makes sure that the external library related Xtext index is persistent and will be available on the next application startup.</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara><emphasis role="strong">External Library Xtext Index Persister</emphasis></simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>This class is responsible for recovering the consistent external library Xtext index state at application startup.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Scheduled on the very first application startup to prepare the Xtext index for the available external libraries.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Recovers the Xtext index state after a force quit and/or application crash.</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara><emphasis role="strong">External Library Preference Page</emphasis></simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>Preference page to configure and update the state of the <emphasis>External Library Preference Store</emphasis>.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Provides a way to install <emphasis>npm</emphasis> dependencies as external libraries into the application.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Reloads the external libraries. Gets the most recent state of N4JS type definition files and updates the Xtext index content based on the current state of the external libraries.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Exports the current npm dependency configuration as a target platform file. This will be discussed in another section ([sec:Headless_External_Library_Support]).</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara><emphasis role="strong">Miscellaneous UI Features</emphasis></simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>Searching for types provided by external libraries.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Opening external modules in read-only editor.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Navigation between external types.</simpara> |
| </listitem> |
| <listitem> |
| <simpara><emphasis>Project Explorer</emphasis> contribution for showing external dependencies for ordinary workspace projects.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Editor-navigator linking support for external modules.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Installing third party npm dependencies directly from package.json editor via a quick fix.</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| </itemizedlist> |
| <section xml:id="subsec:External_Resources"> |
| <title>External Resources</title> |
| <simpara>This approach provides a very pragmatic and simple solution to support external libraries in both in the <literal>IN4JSCore</literal> and in the <literal>IBuilderState</literal>. While <literal>IN4JSCore</literal> supports a completely transparent way of external libraries via the <literal>IN4JSProject</literal> interface all over in the application, the <literal>IBuilderState</literal> is responsible for keeping the Xtext index content up to date with the external libraries. Below picture depicts the hierarchy between the ordinary <literal>IResource</literal> and the <literal>IExternalResource</literal> instances. As described above each external resource is backed by a <literal>java.io.File</literal> resource and each access and operation being invoked on the <literal>IResource</literal> interface will be delegated to this backing resource.</simpara> |
| <figure xml:id="fig:External_Resources_Hierarchy" role="center"> |
| <title>External Resources Hierarchy</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/20_externalLibraries/images/externalResources.svg"/> |
| </imageobject> |
| <textobject><phrase>externalResources</phrase></textobject> |
| </mediaobject> |
| </figure> |
| </section> |
| <section xml:id="subsec:External_Library_Workspace"> |
| <title>External Library Workspace</title> |
| <simpara>External library workspace is an extension of the <literal>InternalN4JSWorkspace</literal>. This workspace is used for storing and managing external libraries all over the application. External libraries can be registered into the workspace by providing one to many external library root folder locations. The provided root folder locations will be visited in an ordered fashion and the contained external libraries (N4JS projects) will be registered into the application. If an external library from a root folder has been registered, then a forthcoming occurrence of an external library with the same artefact identifier (and same folder name) will be ignored at all. For instance let assume two external library root locations are available <literal>ER1</literal> and <literal>ER2</literal>, also <literal>ER1</literal> contains <literal>P1</literal> and <literal>P2</literal> external libraries, while <literal>ER2</literal> contains <literal>P2</literal> and <literal>P3</literal>. After registering the two roots into the workspace <literal>ER1</literal> will be processed first, and <literal>P1</literal> and <literal>P2</literal> will be registered to the workspace, when processing the forthcoming <literal>ER2</literal> root, <literal>P2</literal> will be ignored at all as an external with the same name exists. Finally <literal>P3</literal> will be registered to the workspace. External libraries cannot be registered directly into the workspace it is done automatically by the <emphasis>External Library Preference Store</emphasis> and by the <emphasis>npm Manager</emphasis>.</simpara> |
| </section> |
| <section xml:id="subsec:External_Library_Preference_Store"> |
| <title>External Library Preference Store</title> |
| <simpara>This persistent cache is used for storing an ordered enumeration of registered external library root folder locations. Whenever its internal state is being persisted after a modification, all registered modification listeners will be synchronously notified about this change. All listeners will receive the store itself with the updated state. There are a couple of registered listeners all over the application listening to store update events but the most important one is the <emphasis>External Library Workspace</emphasis> itself. After receiving an external library preference store update event, the external library workspace will calculate the changes from its own state: creates a sort of difference by identifying added, removed and modified external libraries. Also tracks external library root location order changes. Once the workspace has calculated the changes<footnote><simpara>Calculates a list of external library projects that have to be build and another list of projects that have to be cleaned.</simpara></footnote> it will interact with the <emphasis>External Library Builder Helper</emphasis> which will eventually update the persisted Xtext index directly through the <literal>IBuilderState</literal>. After the Xtext index content update all ordinary workspace projects that directly depend either on a built or a cleaned external library will be automatically rebuilt by the external library workspace.</simpara> |
| </section> |
| <section xml:id="subsec:npm_Manager"> |
| <title>Library Manager</title> |
| <simpara>This service is responsible for downloading, installing third party npm dependencies into the local file system. This is done directly by <literal>npm</literal> from <literal>Node.js</literal>. Once an npm package has been downloaded and installed it will be registered into the external library workspace. As part of the registration, the Xtext index content will be updated and all dependent ordinary workspace projects will be rebuilt automatically. An npm package cannot be installed via the <emphasis>Library Manager</emphasis> if it already installed previously.</simpara> |
| </section> |
| <section xml:id="subsec:External_Library_Builder_Helper"> |
| <title>External Library Builder</title> |
| <simpara>This builder is responsible for updating the persisted Xtext index state with external library content directly through the <literal>IBuilderState</literal>. When providing a subset of external libraries to either build or clean, internally it orders the provided external libraries based on the project dependencies. Also, it might skip building all those external libraries that have are being shadowed by a workspace counterpart. An external library is being shadowed by an ordinary workspace project, if the workspace project is accessible and has exactly the same project name as the external library.</simpara> |
| </section> |
| <section xml:id="subsec:External_Library_Xtext_Index_Persister"> |
| <title>External Library Xtext Index Persister</title> |
| <simpara>By default Xtext provides a way to fix corrupted index or to recreate it from scratch in case of its absence. Such inconsistent index states could occur due to application crashes or due to non-graceful application shutdowns. Although this default recovery mechanism provided by Xtext works properly, it is provided only for projects that are available in the Eclipse based workspace (<literal>org.eclipse.core.resources.IWorkspace</literal>) but non of the external libraries are not available from the Eclipse based workspace, so inconsistent external library index content cannot be recovered by this default mechanism. N4JS IDE contributes its own logic to recover index state of external N4JS libraries. When the default Xtext index recovery runs, then it will trigger a external reload as well. This external reload is guaranteed to run always after the default recovery mechanism.</simpara> |
| </section> |
| <section xml:id="subsec:External_Library_Preference_Page"> |
| <title>External Library Preference Page</title> |
| <simpara>This preference page provides a way to configure the external libraries by adding and removing external library root folders, also allows the user to reorder the configured external library root locations. Besides that, npm packages can be installed into the application as external libraries. Neither removing nor reordering built-in external libraries are supported, hence these operations are disabled for built-ins on the preference page. No modifications will take effect unless the changes are persisted with the <literal>Apply</literal> button. One can reset the configurations to the default state by clicking on the <literal>Restore Defaults</literal> button then on the <literal>Apply</literal> button. The <literal>Reload</literal> button will check whether new type definition files are available for npm dependencies, then reloads the persistent Xtext index content based on the available external libraries. Once the external library reloading has been successfully finished, all dependent workspace projects will be rebuilt as well. From the preference page one can export the installed and used third party npm packages as a target platform. This exported target platform file can be used with the headless compiler. After setting up the headless compiler with this exported target platform file, the headless tool will collect and download all required third party npm dependencies.</simpara> |
| </section> |
| </section> |
| <section xml:id="sec:Headless_External_Library_Support" role="language-n4js"> |
| <title>Headless External Library Support</title> |
| <simpara>The headless compiler is not capable of supporting built-in libraries. The whole build and Xtext index creation infrastructure is different in the IDE and in the headless case. Also, due to its archive nature (<literal>n4jsc.jar</literal>) of the headless tool, neither the runtime nor the <literal>Mangelhaft</literal> libraries can be loaded into the headless compiler.</simpara> |
| <simpara>The headless compiler supports downloading, installing and using third party <literal>npm</literal> packages. To enable this feature one has to configure the target platform via the <literal>–targetPlatformFile</literal> (or simply <literal>-tp</literal>) and the <literal>–targetPlatformInstallLocation</literal> (or simply <literal>-tl</literal>) arguments.</simpara> |
| <simpara>If the target platform file argument is configured, then all third party dependencies declared in the target platform file will be downloaded, installed and made available for all the N4JS projects before the compile (and run) phase. If the target platform file is given but the target platform install location is not specified (via the <literal>–targetPlatformInstallLocation</literal> argument), then a the compilation phase will be aborted and the execution will be interrupted.</simpara> |
| <simpara>For more convenient continuous integration and testing purposes there are a couple of additional exception cases with respect to the the target platform file and location that users of the headless compiler have to keep in mind. These are the followings:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><literal>–targetPlatformSkipInstall</literal>. Usually dependencies defined in the target platform file will be installed into the folder defined by option <literal>–targetPlatformInstallLocation</literal>. If this flag is provided, this installation will be skipped, assuming the given folder already contains the required files and everything is up-to-date. Users have to use this flag with care, because no checks will be performed whether the location actually contains all required dependencies.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>If <literal>–targetPlatformSkipInstall</literal> is provided the <literal>–targetPlatformInstallLocation</literal> parameter is completely ignored.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>If <literal>–targetPlatformSkipInstall</literal> is provided the <literal>–targetPlatformFile</literal> parameter is completely ignored.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>If neither <literal>–targetPlatformInstallLocation</literal> not <literal>–targetPlatformFile</literal> parameters are specified the headless tool will treat this case as an implicit <literal>–targetPlatformSkipInstall</literal> configuration.</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>If the target platform install location is configured, and the target platform file is given as well, then all third party dependencies specified in the target platform file will be downloaded to that given location. If the target platform file is given, but the target platform install location is not specified, then a the compilation phase will be aborted and the execution will be interrupted.</simpara> |
| <programlisting language="bash" linenumbering="unnumbered">java -jar n4jsc.jar -projectlocations /path/to/the/workspace/root -t allprojects -tp /absolute/path/to/the/file -tl /path/to/the/target/platform/install/location -rw nodejs -r moduleToRun</programlisting> |
| <section xml:id="_custom-npm-settings"> |
| <title>Custom npm settings</title> |
| <simpara>In some cases there is a need for custom npm settings, e.g. custom npm registry. Those kind of configurations are |
| supported via <literal>.npmrc</literal> file (see <link xl:href="https://docs.npmjs.com/files/npmrc">https://docs.npmjs.com/files/npmrc</link>).</simpara> |
| <simpara>In N4JSIDE user can specify path to his custom configuration file in the preference page.</simpara> |
| <simpara>For the commandline N4JSC.jar provides special option <literal>-npmrcRootLocation</literal> that allows headless compiler to |
| use custom settings.</simpara> |
| </section> |
| </section> |
| <section xml:id="sec:lmFutureWork"> |
| <title>Future Work</title> |
| <simpara>Some aspects not covered in current design, but worth consideration in the future</simpara> |
| <section xml:id="subsec:lmMultipleDependencyScope"> |
| <title>Multiple Dependency Scope</title> |
| <simpara>npm scope dependencies</simpara> |
| <variablelist> |
| <varlistentry> |
| <term><emphasis role="strong">DEPENDENCY_DEVELOPMENT</emphasis> </term> |
| <listitem> |
| <simpara><link xl:href="https://docs.npmjs.com/files/package.json#devdependencies">https://docs.npmjs.com/files/package.json#devdependencies</link></simpara> |
| </listitem> |
| </varlistentry> |
| <varlistentry> |
| <term><emphasis role="strong">DEPENDENCY_PEER</emphasis> </term> |
| <listitem> |
| <simpara><link xl:href="https://docs.npmjs.com/files/package.json#peerdependencies">https://docs.npmjs.com/files/package.json#peerdependencies</link></simpara> |
| </listitem> |
| </varlistentry> |
| <varlistentry> |
| <term><emphasis role="strong">DEPENDENCY_BUNDLE</emphasis> </term> |
| <listitem> |
| <simpara><link xl:href="https://docs.npmjs.com/files/package.json#bundleddependencies">https://docs.npmjs.com/files/package.json#bundleddependencies</link></simpara> |
| </listitem> |
| </varlistentry> |
| <varlistentry> |
| <term><emphasis role="strong">DEPENDENCY_OPTIONAL</emphasis> </term> |
| <listitem> |
| <simpara><link xl:href="https://docs.npmjs.com/files/package.json#optionaldependencies">https://docs.npmjs.com/files/package.json#optionaldependencies</link></simpara> |
| </listitem> |
| </varlistentry> |
| <varlistentry> |
| <term><emphasis role="strong">DEPENDENCY_PROVIDES</emphasis> </term> |
| <listitem> |
| <simpara><link xl:href="http://www.rpm.org/wiki/PackagerDocs/Dependencies#Provides">http://www.rpm.org/wiki/PackagerDocs/Dependencies#Provides</link></simpara> |
| </listitem> |
| </varlistentry> |
| <varlistentry> |
| <term><emphasis role="strong">DEPENDENCY_WEAK</emphasis> </term> |
| <listitem> |
| <simpara><link xl:href="http://www.rpm.org/wiki/PackagerDocs/Dependencies#Weakdependencies">http://www.rpm.org/wiki/PackagerDocs/Dependencies#Weakdependencies</link></simpara> |
| </listitem> |
| </varlistentry> |
| </variablelist> |
| </section> |
| <section xml:id="subsec:lmRunTestsFromLibrary"> |
| <title>Run Tests from TestLibrary</title> |
| <simpara>Imagine we are implementing some API, and we want to run tests for that API. Tests are delivered to us as separate package, and there is not direct association between implementation and test projects (tests are not depending on implementation). Still we want to run provided tests to see if our implementation complies with API tests, e.g. AcceptanceTest suite for Application written against application sdk.</simpara> |
| </section> |
| </section> |
| </chapter> |
| <chapter xml:id="sec:JSON_Support"> |
| <title>JSON Support</title> |
| <warning> |
| <simpara>This chapter may be outdated.</simpara> |
| </warning> |
| <section xml:id="sec:JSON_Parser"> |
| <title>JSON Parser</title> |
| <simpara>For the JavaScript Object Notation format, a multitude of specifications exist (e.g. [<link linkend="RFC8259">RFC8259</link>], [<link linkend="ECMA404">ECMA404</link>], [<link linkend="RFC7158">RFC7158</link>]). While all specifications agree on the basic structure of JSON, many special cases exist, which remain un-addressed by the specifications. Therefore, in practice many parsers exhibit implementation-specific behavior. See [<link linkend="Ser18">Ser18</link>] for a discussion of many of these cases.</simpara> |
| <simpara>The initial grammar of the N4JS JSON parser is mostly based on [<link linkend="ECMA404">ECMA404</link>] while further adaptions have been made to accommodate for special cases such as escaping unicode control characters in string literals. This section discusses the different aspect in which our parser exhibits special behavior in order to parse a maximum of real-world JSON text. If applicable, we will also discuss differences between JSON and the N4JS syntax for object literals.</simpara> |
| <section xml:id="sec:JSON_Parser_Unicode_Escaping"> |
| <title>Escaping Unicode Control Characters in String Literals</title> |
| <simpara>In comparison to ECMAScript (and thus N4JS), JSON only requires the escaping of unicode control characters in the range from <literal>U+0000</literal> to <literal>U+001F</literal> when used in a string literal. This includes the line termination characters <literal>U+000A</literal> or <literal>\n</literal> and <literal>U+000D</literal> or <literal>\r</literal>. However, different from ECMAScript, JSON allows the unescaped use of the unicode characters <literal>U+2028</literal> and <literal>U+2029</literal> within string literals. Therefore, the JSON parser differs from the behavior of the N4JS parser.</simpara> |
| </section> |
| <section xml:id="sec:JSON_Parser_Empty_Text"> |
| <title>Empty Text</title> |
| <simpara>While the abovementioned JSON specifications do not allow for empty JSON text (e.g. no data, only whitespace data), our parser is tolerant towards such inputs. The reason behind this decision is that it allows users of the N4JS IDE, to create new empty JSON files without experiencing any parser errors.</simpara> |
| </section> |
| <section xml:id="sec:JSON_Parser_Nested_Structures"> |
| <title>Nested Structures</title> |
| <simpara>Since the N4JS JSON parser is implemented in terms of a recursive decent parser, the parsable inputs are limited in terms of nesting. This results in a stack overflow exception for highly nested input data.</simpara> |
| </section> |
| <section xml:id="sec:JSON_Parser_Whitespace"> |
| <title>Whitespace</title> |
| <simpara>[<link linkend="RFC7158">RFC7158</link>] and [<link linkend="ECMA404">ECMA404</link>] both define whitespace characters of JSON to be exclusively <literal>U+0009</literal>, <literal>U+000A</literal>, <literal>U+000D</literal> and <literal>U+0020</literal>. However, for now we adopt the whitespace strategy of the N4JS parser, which allows for additional whitespace characters (also see [<link linkend="ECMA15a">ECMA15a</link>] section 7.2 White Space). This only applies to JSON text outside of string literals.</simpara> |
| </section> |
| <section xml:id="sec:JSON_Parser_Comments"> |
| <title>Comments</title> |
| <simpara>Although JSON as a standard does not specify any notion of source code comments, we allow them on a parser level. (single line comments introduced by <literal>//</literal> and multi-line comments using <literal>/*</literal> and <literal>*/</literal>). After parsing, we issue corresponding validation messages that indicate to the user, that such comments will possibly not be parsable in a different context (e.g. by npm in case of package.json files).</simpara> |
| </section> |
| </section> |
| <section xml:id="sec:JSON_Language_Extensions"> |
| <title>JSON Language Extensions</title> |
| <simpara>Generally, our JSON support was implemented with generic JSON support in mind. However, for some types of JSON files (e.g. <literal>package.json</literal> files) we provide custom behavior to better assist the user. This includes custom JSON validators and resource description strategies that only apply to certain types of JSON files. In order to keep our JSON implementation independent from N4JS-specific code, these concerns are separated using an Eclipse Extension Point. In the headless case, extension also need to be registered manually via the <literal>JSONExtensionRegistry</literal>. In the following we present the different aspect in which the JSON language can be extended.</simpara> |
| <section xml:id="sec:JSON_Validator_Extensions"> |
| <title>JSON Validator Extensions</title> |
| <simpara>Via the JSON validator extensions (cf. <literal>IJSONValidatorExtension</literal>), other bundles can register JSON validator extensions that introduce domain-specific validation for specific types of JSON files.</simpara> |
| <section xml:id="sec:File_Specitic_Validator_Extensions"> |
| <title>File-Specific Validator Extensions</title> |
| <simpara>For every JSON resource that is validated, all registered JSON validator extensions are executed. For a validator extension to be specific to a certain type of file, one may override the Xtext method <literal>isResponsible</literal> which checks whether a validator applies on a per resource basis.</simpara> |
| </section> |
| <section xml:id="sec:JSON_Declarative_JSON_Validator_Extensions"> |
| <title>Declarative JSON Validator Extensions</title> |
| <simpara>In addition to Xtext’s <literal>@Check</literal> methods, we provide an annotation <literal>@CheckProperty</literal> that allows to declare JSON-specific check methods that only apply to certain property paths of a JSON document. The <literal>@CheckProperty</literal> annotation can only be used when inheriting from the <literal>AbstractJSONValidatorExtension</literal> class. The following example outlines its usage:</simpara> |
| <programlisting language="java" linenumbering="unnumbered">(...) |
| @CheckProperty(propertyPath = "prop1") |
| public void checkProperty(JSONValue propertyValue) { |
| // validate JSONValue for top-level property 'prop1' |
| } |
| |
| @CheckProperty(propertyPath = "nested.prop2") |
| public void checkNestedProperty(JSONValue propertyValue) { |
| // validate JSONValue for the nested property 'nested.prop2' |
| } |
| (...)</programlisting> |
| <simpara>The examples illustrates how property-check-method can be declared. In a JSON document the first check method will only be invoked for the JSON value that is set for top-level property <literal>prop1</literal>. The second check method is invoked for the nested property <literal>prop2</literal> of the object that is set for the top-level property <literal>prop2</literal>.</simpara> |
| <programlisting language="json" linenumbering="unnumbered">{ |
| "prop1": 1, // checkProperty applies |
| "nested": { |
| "prop2": 2 // checkNestedProperty applies |
| } |
| }</programlisting> |
| <simpara>When inheriting from the <literal>AbstractJSONValidatorExtension</literal>, the usual <literal>addIssue</literal> methods may be used in order to issue validation messages. Furthermore, the issue codes that are being used for the messages may originate from the bundle that hosts the validator extension.</simpara> |
| </section> |
| </section> |
| <section xml:id="_json-resource-description-strategy"> |
| <title>JSON Resource Description Strategy</title> |
| <simpara>Via JSON resource description extensions (cf. <literal>IJSONResourceDescriptionExtension</literal>), other bundles can define a custom resource description strategy for specific types of JSON files. As a consequence, these extensions define what information on the contents of a JSON file is stored in the Xtext index (cf. <literal>IResourceDescriptions</literal>).</simpara> |
| </section> |
| </section> |
| </chapter> |
| <chapter xml:id="_jsdoc"> |
| <title>JSDoc</title> |
| <warning> |
| <simpara>This chapter may be outdated.</simpara> |
| </warning> |
| <section xml:id="sec:Design_Rationale" role="language-n4js"> |
| <title>Design Rationale</title> |
| <simpara>JSDoc parser provides general API for parsing and obtaining information embedded in comments. This may have low significance for N4JS itself but is essential when working with JS dialects (e.g. vanilla JS) that use JSDoc comments for enrich given dialect with semantic information. Anticipated uses may include: - type information extraction when importing external code - validation of links between commented fragments - supporting markup for documentation</simpara> |
| <note> |
| <simpara>Although current focus is on migration process we want to provide general solution that can be customized for given use cases</simpara> |
| </note> |
| <section xml:id="_general-design"> |
| <title>General Design</title> |
| <simpara>When using the API client has to create instance of DocletParser and configure it with <emphasis>LienTag</emphasis>s and <emphasis>InLineTag</emphasis>s that are to be used.</simpara> |
| <simpara>LineTags can depend on InlineTags (e.g. InlineTg in description of parameters of the LineTag). API provides support for that but client has to configure this.</simpara> |
| <figure xml:id="fig:cd_JSDocParserAPI" role="center"> |
| <title>JSDoc Parser API (without model)</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/23_jsdoc/images/cd_JSDocParserAPI.svg"/> |
| </imageobject> |
| <textobject><phrase>cd JSDocParserAPI</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara>Initiated DocletParser can be used to parse <emphasis>String</emphasis> containing given JSDoc comment.Based on provided <emphasis>ITagDefinition</emphasis>s parser will parse input string and return JSDoc DOM AST (see fig. XX with ecore diagram). By querying tree structure client can obtain information extracted form parsed input String.</simpara> |
| <figure xml:id="fig:cd_JSDocModel" role="center"> |
| <title>JSDoc AST/DOM Model</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/23_jsdoc/images/cd_JSDocModel.svg"/> |
| </imageobject> |
| <textobject><phrase>cd JSDocModel</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara>The root of the this AST is the Doclet. The doclet itself contains some content (as it is a Composite containing ContentNodes), usually Description and an arbitrary number of LineTags. LineTags are created by custom ITagDefinition implementations that extend AbstractLineTagDefinition. They contain the title and an arbitrary number of values, stored in a map. These values are Composite nodes as well, containing tag specific content. E.g., the parameter tag will create the values for the parameter type, the parameter name, and the description. Description (in LineTags as well as in doclet itself) is free text with optional InlineTags. InlineTags are created by custom ITagDefinition that extends AbstractInlineTagDefinition. In general tag values are designed as a comprise between a very general tree (basically containing only text nodes) and a structured typed tree (containing type expressions etc.). Although it should be possible to model a tag value by only using Text nodes, some special typed nodes are provided for sake of simplicity (and probably performance). E.g., TypeReferences are modeled using a typed node, in order to simplify external handling of these references (for type analysis, and refactorings such as renaming a type). New typed nodes will probably be introduced (see below). Markers can be attached to nodes for internal purposes. E.g., a marker can be used to distinguish different syntax versions of defining an array (<literal>String[]</literal> vs. <literal>Array<String></literal>).Literals can be used to simplify rewriting. That is, they can contain information on line breaks, JSDoc line prefixes etc., which are not needed for the semantics of a tag, but only for the syntax (not used yet). Also, each node contains a position for simplifying rewriting.</simpara> |
| <figure xml:id="fig:cd_JSDocParserAndModel" role="center"> |
| <title>JSDoc Parser and Model</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/23_jsdoc/images/cd_JSDocParserAndModel.svg"/> |
| </imageobject> |
| <textobject><phrase>cd JSDocParserAndModel</phrase></textobject> |
| </mediaobject> |
| </figure> |
| </section> |
| <section xml:id="sec:Type_Expressions"> |
| <title>Type Expressions</title> |
| <simpara>Type expressions are are not handled by JSDoc Parser by itself as instances of the grammars are use case specific, e.g. for migration purposes grammar for type expressions used in migration process is specific to the in question therefore specific type expressions parser will be provided separately.DocLetParser by default can only extract String representing given type expression, parsing and interpreting it stays in responsibility of given tag implementation provided by the client.</simpara> |
| </section> |
| </section> |
| </chapter> |
| <chapter xml:id="_docexporter"> |
| <title>DocExporter</title> |
| <warning> |
| <simpara>This chapter may be outdated.</simpara> |
| </warning> |
| <simpara>The DocExporter exports JavaDoc from source files to adoc files. |
| In particular it combines information about methods with tests in order to create specification documents.</simpara> |
| <section xml:id="sec:Specification_Exporter"> |
| <title>Specification Exporter</title> |
| <simpara>The specification exporter creates and merges artifacts. |
| <link linkend="fig:api_test_spec">Fig. API Test Spec</link> sketches the relation between API (i.e., n4jsd files with classifiers), |
| tests (i.e., N4JS test classes and methods), and specification (Documentation with JSDOC markers).</simpara> |
| <figure xml:id="fig:api_test_spec" role="center"> |
| <title>Component/Class pseudo diagram: Relation API, Tests and Specification</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/24_docexporter/images/api_test_spec.svg"/> |
| </imageobject> |
| <textobject><phrase>api test spec</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara><link linkend="fig:cd_jsdocreader">Fig. Exporter</link> shows the classes that read and analyze Java source documentation.</simpara> |
| <figure xml:id="fig:cd_jsdocreader" role="center"> |
| <title>Java reader classes</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/24_docexporter/images/cd_jsdocreader.svg"/> |
| </imageobject> |
| <textobject><phrase>cd jsdocreader</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara><link linkend="fig:cd_adocexporter">Fig. Exporter</link> shows the content classes of the exporter that contain the generated documentation contents.</simpara> |
| <figure xml:id="fig:cd_adocexporter" role="center"> |
| <title>Exporter content classes</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/24_docexporter/images/cd_adocexporter.svg"/> |
| </imageobject> |
| <textobject><phrase>cd adocexporter</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara><link linkend="fig:cd_docexporter_model">Fig. DocExporter Model</link> shows the model, which may be used for other doc exporters as well.</simpara> |
| <figure xml:id="fig:cd_docexporter_model" role="center"> |
| <title>Exporter model classes</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/24_docexporter/images/cd_docexporter_model.svg"/> |
| </imageobject> |
| <textobject><phrase>cd docexporter model</phrase></textobject> |
| </mediaobject> |
| </figure> |
| </section> |
| </chapter> |
| <chapter xml:id="_rename-refactoring"> |
| <title>Rename Refactoring</title> |
| <simpara>The rename refactoring operation is implemented based on current Xtext’s rename refactoring implementation. However, lots of customization have been done in order to make Rename Refactoring work for N4JS. In order to understand N4JS customization, it is imperative to understand how Xtext implements rename refactoring. In this chapter, we will focus on Xtext’s architecture for rename refactoring. Additionally, we will point to the components that have been customized for N4JS.</simpara> |
| <section xml:id="_rename-refactoring-ui-interaction"> |
| <title>Rename Refactoring UI interaction</title> |
| <simpara>Xtext’s implementation allows rename refactoring be in either one of two modes (1) Direct refactoring mode (3) Refactoring with dialog mode. Diagram <link linkend="fig:rename_refactoring_communication_diagram_part1">Direct Rename Refactoring UI interaction</link> shows the UI interaction in <emphasis>direct refactoring mode</emphasis>.</simpara> |
| <informalfigure xml:id="fig:rename_refactoring_communication_diagram_part1" role="center"> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/25_renameRefactoring/images/rename_refactoring_communication_diagram_part1.svg"/> |
| </imageobject> |
| <textobject><phrase>title-"Direct Rename Refactoring UI interaction"</phrase></textobject> |
| </mediaobject> |
| </informalfigure> |
| <simpara>In this diagram, the classes in yellow are customized by N4JS implementation to handle N4JS-specific characteristics.</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><literal>DefaultRenameElementHandler</literal>: Our custom N4JS implementation converts the selected element to be renamed into its corresponding TModule element.</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>ILinkedPositionGroupCalculator</literal>: This class is responsible for calculating locations of names to be changed during linked edit mode. We need to provide a custom N4JS implementation to handle composed elements.</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>RenameElementProcessor</literal>: We need to provide a custom N4JS implementation to add N4JS-specific validation of conditions, e.g. checking name conflicts etc.</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>The key class for creating updates of a declaration and its associated references is <literal>RenameElementProcessor</literal>. In the following section, we will see how this class interacts with other classes to achieve this.</simpara> |
| </section> |
| <section xml:id="_renameelementprocessor-interaction"> |
| <title>RenameElementProcessor interaction</title> |
| <simpara>Diagram <link linkend="fig:rename_refactoring_communication_diagram_part2">RenameElementProcessor interaction</link> shows the interaction of <literal>RenameElementProcessor</literal> and other classes to create changes for both declaration and references during rename refactoring.</simpara> |
| <informalfigure xml:id="fig:rename_refactoring_communication_diagram_part2" role="center"> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/25_renameRefactoring/images/rename_refactoring_communication_diagram_part2.svg"/> |
| </imageobject> |
| <textobject><phrase>title-"RenameElementProcessor interaction"</phrase></textobject> |
| </mediaobject> |
| </informalfigure> |
| <simpara>As seen in the diagram, there are two stages of creating updates:</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>Creating updates for declaration is done by <literal>IRenameStrategy</literal> and</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Creating updates for references is done by <literal>ReferenceUpdateDispatcher</literal>. <literal>ReferenceUpdateDispatcher</literal> in turn delegates the finding of references to <literal>IReferenceFinder</literal>.</simpara> |
| </listitem> |
| </orderedlist> |
| <simpara>The text edits for changing the definition and the references are accumulated by an <literal>IRefactoringUpdateAcceptor</literal>.</simpara> |
| <simpara>The classes in yellow are customized by N4JS implementation.</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><literal>IRenameStrategy</literal>: the custom N4JS implementation creates updates for constituent members of composed elements.</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>IReferenceFinder</literal>: the custom N4JS implementation used for finding references of a declaration.</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>RefactoringCrossReferenceSerializer</literal>: custom N4JS implementation to retrieve the updated name for cross references. For some unknown reason, the default implementation does not work correctly.</simpara> |
| </listitem> |
| </itemizedlist> |
| </section> |
| </chapter> |
| <chapter xml:id="chap:flowgraphs"> |
| <title>Flow Graphs</title> |
| <section xml:id="sec:flowgraphs_overview" role="language-n4js"> |
| <title>Flow graphs overview</title> |
| <simpara>In this chapter, the control and data flow analyses are introduced. |
| Since not all AST elements are relevant for the control or data flow analyses, a new marker class is introduced called <literal>ControlFlowElement</literal>. |
| All AST elements which are part of the control flow graph implement this class. |
| The term control flow is abbreviated as <emphasis>CF</emphasis> and hence <literal>ControlFlowElement</literal>s are abbreviated as <emphasis>CF elements</emphasis>.</simpara> |
| <simpara></simpara> |
| <sidebar> |
| <simpara><link xl:href="https://github.com/eclipse/n4js/issues/120"><inlinemediaobject> |
| <imageobject> |
| <imagedata fileref="images/issue.svg"/> |
| </imageobject> |
| <textobject><phrase></phrase></textobject> |
| </inlinemediaobject> #120</link></simpara> |
| </sidebar> |
| <simpara>The following picture shows the control flow graph of the function <literal>f</literal>.</simpara> |
| <figure> |
| <title>Control flow graph of a simple function</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/30_flowgraphs/images/cfg_for_loop.png"/> |
| </imageobject> |
| <textobject><phrase>cfg for loop</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <section xml:id="_internal-graph"> |
| <title>Internal graph</title> |
| <simpara>Every <emphasis>internal graph</emphasis> refers to a single <emphasis>control flow container</emphasis>. |
| The graph consists of <emphasis>nodes</emphasis> and <emphasis>edges</emphasis>, where the edges are instances of <literal>ControlFlowEdge</literal>, and nodes are instances of <literal>Node</literal>. |
| Additionally, a so called <emphasis>complex node</emphasis> is used to group nodes that belong to the same CF element.</simpara> |
| <variablelist> |
| <varlistentry> |
| <term>Internal graph</term> |
| <listitem> |
| <simpara>Control flow graphs are created based on the AST elements. |
| Nevertheless, a fine-grained abstraction is used that is called <emphasis>internal graph</emphasis>. |
| The internal graph reflects all control flows and data effects that happen implicitly and are part of the language’s semantics. |
| For instance, the for-of statement on iterable objects forks the control flow after invoking the <literal>next()</literal> method. |
| This is done implicitly and not part of the written source code. |
| Moreover, this invocation could cause side effects. |
| These control flows and effects are reflected using the internal graph. |
| To implement analyses that refer to AST elements, an API for flow analyses is provided which hides the internal graph and works with AST elements only. |
| In the following, the term <emphasis>control flow graph</emphasis> refers to the internal graph.</simpara> |
| </listitem> |
| </varlistentry> |
| <varlistentry> |
| <term>Control flow container</term> |
| <listitem> |
| <simpara>At several places in the AST, an execution of statements or elements can happen. |
| Obviously, statements can be executed in bodies of methods or function expressions. |
| In addition, execution can also happen in field initializers or the <literal>Script</literal> itself. |
| Since all these AST elements can contain executable control flow elements (CF elements), they thus contain a control flow graph. |
| In the following, these AST elements are called <emphasis>control flow containers</emphasis> or CF containers.</simpara> |
| </listitem> |
| </varlistentry> |
| <varlistentry> |
| <term>Nodes</term> |
| <listitem> |
| <simpara>Nodes represent complete CF elements or parts of them. |
| For instance, simple CF elements like a <literal>break</literal> statement are represented using only one node. |
| Regarding more complex CF elements that introduce a more complex control flow, due to e.g. nested expressions, several nodes represent one CF element. |
| All nodes of a single CF element are grouped within a complex node.</simpara> |
| </listitem> |
| </varlistentry> |
| <varlistentry> |
| <term>Edges</term> |
| <listitem> |
| <simpara>Edges reference a start and an end node which reflects a forward control flow direction, which is in the following called forward traverse direction. |
| Traversing from end to start of an edge is thus called backward traverse direction. |
| The so called <emphasis>next node</emphasis> is either the end node in context of forward, or the start node in context of backward traverse direction. |
| Edges also reflect the reason of the control flow using a control flow type. |
| The default control flow type is called <literal>Successor</literal> and such edges connect two ordinary subsequent nodes. |
| Other types like <literal>Return</literal> or <literal>Break</literal> indicate control flow that happens due to return or break statements. |
| A special control flow type is <literal>Repeat</literal> that indicates the entry of a loop body. |
| This edge is treated specially when traversing the control flow graph to avoid infinitive traversals of loops.</simpara> |
| </listitem> |
| </varlistentry> |
| <varlistentry> |
| <term>Complex node</term> |
| <listitem> |
| <simpara>The complex node always has a single entry and exit node, no matter the control flow it causes. |
| For instance, although the for-statement can contain various control flows among its nested CF elements, its complex node still has a single entry and exit node. |
| This simplifies concatenating subsequent complex nodes, since only their exit nodes have to be connected to the following entry node. |
| Aside from exit and entry node, a complex node usually contains additionally nodes to represent the CF element. |
| These nodes are connected by control flow edge so that their control flow lies within complex node. |
| However, regarding nested CF elements, the control flow leaves and re-enters a complex node. |
| To specify which CF element is nested, a delegating node (<literal>DelegatingNode</literal>) is created that points to the nested CF element.</simpara> |
| </listitem> |
| </varlistentry> |
| </variablelist> |
| <simpara>Consider that source code elements can be nested like expressions that have sub-expressions as in <literal>1 + 2 * 3</literal>. |
| Also statements can contain other statements like in <literal>if (true) return;</literal>. |
| The design of the control flow graph deals with this nesting by mapping CF elements to several nodes. |
| All nodes of one CF element are aggregated into the <emphasis>complex node</emphasis>.</simpara> |
| <figure xml:id="img:internalGraph"> |
| <title>The source code of <literal>1+2</literal> creates an internal graph of three complex nodes to deal with nested integer literals</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/30_flowgraphs/images/internalGraph.png"/> |
| </imageobject> |
| <textobject><phrase>internalGraph</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara>The example in the <link linkend="img:internalGraph">figure above</link> shows the internal graph produced by the source code <literal>1+2</literal>. |
| Additionally, a simpler version of the internal graph is shown (called <emphasis>User Graph View</emphasis>), with which client analyses deal. |
| The user graph view is only a view on the internal graph, but does not exist as an own instance. |
| In the figure, the nesting of the integer literals becomes obvious: |
| The control flow edges of delegating nodes targets entry nodes of different CF elements. |
| Also, there are CF edges from the exit nodes of these nested CF elements to return the control flow.</simpara> |
| <figure xml:id="img:cn_for_stmt"> |
| <title>Complex Node of for statement</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/30_flowgraphs/images/cn_for_stmt.png"/> |
| </imageobject> |
| <textobject><phrase>cn for stmt</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara>In the <link linkend="img:cn_for_stmt">above figure</link>, the complex node of the for statement is shown. |
| The details of the complex nodes of the nested CF elements (such as the initializer or the body statement) are omitted. |
| The figure displays the control flow fork after the condition and also shows the <emphasis>Repeat</emphasis> edge that targets the for body. |
| The node called <emphasis>Catch Node</emphasis> is used in situations when there are jumping control flows introduced for instance by a continue statement. |
| The catch will will then be the target of an control flow edge that starts at the continue statement.</simpara> |
| <simpara>The graph of nodes and edges is constructed in the following order.</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>First, for every CF element a complex node and all its nodes are created. |
| Also, the nodes within the complex node are connected according to their control flow behavior.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Second, all subsequent complex nodes are connected by connecting their exit and entry nodes. |
| Moreover, nested complex nodes are connected by interpreting the delegating nodes.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Third, jumping control due to <literal>return</literal>, <literal>throw</literal>, <literal>break</literal> or <literal>continue</literal> statements is computed. |
| This is done by first deleting the successor edge of the jumping statement and introducing a new control flow edge that ends at the jump target.</simpara> |
| </listitem> |
| </itemizedlist> |
| </section> |
| <section xml:id="_optimizations"> |
| <title>Optimizations</title> |
| <simpara>The internal graph contains many nodes to simplify the graph construction. |
| However, these nodes carry no relevant information when traversing the graph. |
| Consequently, in an optimization step, they are removed from the graph for performance reasons.</simpara> |
| <simpara>A node removal for a node <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>n</mi><mn>2</mn></msub></math> is done by replacing the path <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>n</mi><mn>1</mn></msub><mo>-</mo><mo>></mo><msub><mi>n</mi><mn>2</mn></msub><mo>-</mo><mo>></mo><msub><mi>n</mi><mn>3</mn></msub></math> by the new path <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>n</mi><mn>1</mn></msub><mo>-</mo><mo>></mo><msub><mi>n</mi><mn>3</mn></msub></math>. |
| These removals are done for delegating nodes that only have one incoming and one outgoing edge.</simpara> |
| <simpara>A second kind but similar optimization reduces the number of helper nodes that are used as entry nodes. |
| In case a complex nodes consists only of exactly one entry and one exit node, both of these nodes are collapsed into one node. |
| This remaining node then is the representing node of the AST element.</simpara> |
| </section> |
| <section xml:id="_api-for-client-analyses"> |
| <title>API for client analyses</title> |
| <simpara>To implement client analyses based on the control flow graph, the three classes <literal>GraphVisitor</literal>, <literal>GraphExplorer</literal> and <literal>BranchWalker</literal> are provided. |
| They provide the means to visit CF elements in a control flow graph and also to traverse single control flow paths. |
| The method <literal>N4JSFlowAnalyses#analyze</literal> can execute several client analyses in one run to maintain scalability.</simpara> |
| <section xml:id="_mapping-from-internal-to-ast-elements"> |
| <title>Mapping from internal to AST elements</title> |
| <simpara>The API classes work with AST elements such as <literal>ControlFlowElement</literal> instead of the internally used graph classes <literal>ComplexNode</literal>, <literal>Node</literal> or <literal>ControlFlowEdge</literal>. |
| The mapping from internal classes to AST elements is done in the <literal>GraphVisitor</literal> class.</simpara> |
| <simpara>Note that the control flow graph has the following properties:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><literal>ExpressionStatement</literal>s are not represented. |
| Instead, only their expressions are represented. |
| Nevertheless, the API can deal with calls that refer to expression statements, e.g. when requesting their successors.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Control statements are also not represented in the graph, but can also be used in calls to the API. |
| The reason is, that it is unclear when a control statement (e.g. a for loop) is visited exactly.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Since a <literal>FlowEdge</literal> which connects two <literal>ControlFlowElement</literal>s can represent multiple internal edges, it can have multiple <literal>ControlFlowType</literal>s.</simpara> |
| </listitem> |
| </itemizedlist> |
| </section> |
| <section xml:id="_graph-visitor"> |
| <title>Graph visitor</title> |
| <simpara>Graph visitors traverse the control flow graphs of every CF container of a script instance in the following two traverse directions:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><emphasis>Forward</emphasis>: from the container’s start to all reachable CF graph elements.</simpara> |
| </listitem> |
| <listitem> |
| <simpara><emphasis>Backward</emphasis>: from the container’s end to all reachable CF graph elements.</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>In each traverse direction, the graph visitor visits every reachable CF element and edge. |
| Note that neither empty statements nor control statements are part of the control flow graph. |
| The order of visited CF elements is related to either a breadth or a depth search on the CF graph. |
| However, no specific order assumptions are guaranteed.</simpara> |
| <figure> |
| <title>The CF elements <literal>"loop"</literal> and <literal>"end"</literal> are dead code and displayed in grey.</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/30_flowgraphs/images/deadcode.png"/> |
| </imageobject> |
| <textobject><phrase>deadcode</phrase></textobject> |
| </mediaobject> |
| </figure> |
| </section> |
| <section xml:id="_graph-explorer"> |
| <title>Graph explorer</title> |
| <simpara>Graph visitors can request a <emphasis>graph explorer</emphasis> to be activated under specific conditions related to the client analysis. |
| A graph explorer is the start point to analyze control flow branches. |
| The first control flow branches is started directly at the graph explorer’s creation site, but of course this first branches might fork eventually. |
| The graph explorer keeps track of all forked branches that originate at its activation site. |
| It also provides the means to join previously forked branches again.</simpara> |
| </section> |
| <section xml:id="_branch-walker"> |
| <title>Branch walker</title> |
| <simpara>With every graph explorer, a branch walker is created that traverses the control flow graph beginning from the activation site of the graph explorer. |
| On every such branch, the two visit methods of CF elements and edges respectively, are called in the order of the traverse direction. |
| Every time the branch forks, the fork method of the branch walker is invoked and creates another branch walker which will continue the traversal on the forked branch. |
| The fork method can be used to copy some path data or state to the newly forked branch walker. |
| Note that every edge is always followed by the branch walker except for repeat edges which are followed exactly twice. |
| The reason to follow them twice is that first, following them only once would hide those control flows that re-visit the same CF elements due to the loop. |
| Second, following them more than twice does not reveal more insights, but only increases the number of branches. |
| When control flow branches merge again, for instance at the end of an <literal>if</literal>-statement, two or more branch walkers are merged into a new succeeding one. |
| The graph explorer provides the means to do this. |
| In case a CF element has no next elements, the branch walker terminates.</simpara> |
| </section> |
| <section xml:id="_example-1-compute-string-for-each-path"> |
| <title>Example 1: Compute string for each path</title> |
| <simpara>Let’s assume that we want to compute all control flow branches of a function and use the client API for that. |
| The function <literal>f()</literal> in the following code snippet has four control flow branches: <literal>1 → 2</literal>, <literal>→ 3 →</literal>, <literal>→ 4 →</literal> and <literal>5</literal>.</simpara> |
| <formalpara> |
| <title>Function <literal>f()</literal> has four control flow branches.</title> |
| <para> |
| <screen>function f() { |
| 1; |
| if (2) |
| 3; |
| else |
| 4; |
| 5; |
| }</screen> |
| </para> |
| </formalpara> |
| <simpara>To compute these control flow branches, the class <literal>AllBranchPrintVisitor</literal> extends the <literal>GraphVisitor</literal>. |
| Already in the method <literal>initializeMode()</literal> a graph explorer is activated. |
| Note that the method <literal>requestActivation()</literal> can be understood as a <literal>addListener</literal> method for a listener that listens to visit events on nodes and edges. |
| Immediately after the activation request, the first branch walker is created in the method <literal>firstBranchWalker()</literal>.</simpara> |
| <simpara>The first visited CF element of the branch walker will then be the expression <literal>1</literal>. |
| It is formed into a string and added to the variable <literal>curString</literal>. |
| After expression <literal>1</literal>, the flow edge from <literal>1</literal> to <literal>2</literal> is visited. |
| This will concatenate the string <literal>→</literal> to the path string. |
| Variable <literal>curString</literal> will eventually hold the branch string like <literal>1 → 2</literal>. |
| Since the control flow forks after <literal>2</literal>, the method <literal>forkPath()</literal> is called and creates two new instances of a branch walker. |
| These new instances succeed the the first branch walker instance and each traverses one of the branches of the <literal>if</literal>-statement. |
| When the <literal>if</literal>-statement is passed, these two branches are merged into a new succeeding branch walker. |
| After all branch walkers are terminated, the graph explorer and graph visitor are also terminated. |
| The method <literal>getBranchStrings()</literal> collects all four computed strings from the variable <literal>curString</literal> of all branch walkers.</simpara> |
| <formalpara> |
| <title>Implementation of a graph visitor that computes all control flow paths</title> |
| <para> |
| <screen>class AllBranchPrintVisitor extends GraphVisitor { |
| protected void initializeMode(Mode curDirection, ControlFlowElement curContainer) { |
| super.requestActivation(new AllBranchPrintExplorer()); |
| } |
| |
| class AllBranchPrintExplorer extends GraphExplorer { |
| class AllBranchPrintWalker extends BranchWalker { |
| String curString = ""; |
| |
| protected void visit(ControlFlowElement cfe) { |
| curString += cfe.toString(); |
| } |
| |
| protected void visit(FlowEdge edge) { |
| curString += " -> "; |
| } |
| |
| protected AllBranchPrintWalker forkPath() { |
| return new AllBranchPrintWalker(); |
| } |
| } |
| |
| protected BranchWalker joinBranches(List<BranchWalker> branchWalkers) { |
| // TODO Auto-generated method stub |
| return null; |
| } |
| |
| protected BranchWalkerInternal firstBranchWalker() { |
| return new AllBranchPrintWalker(); |
| } |
| } |
| |
| List<String> getBranchStrings() { |
| List<String> branchStrings = new LinkedList<>(); |
| for (GraphExplorerInternal app : getActivatedExplorers()) { |
| for (BranchWalkerInternal ap : app.getAllBranches()) { |
| AllBranchPrintWalker printPath = (AllBranchPrintWalker) ap; |
| branchStrings.add(printPath.curString); |
| } |
| } |
| return branchStrings; |
| } |
| }</screen> |
| </para> |
| </formalpara> |
| </section> |
| <section xml:id="_path-quantor"> |
| <title>Path quantor</title> |
| <simpara>Graph explorers are typically used to reason on all branch walkers that start at a specific location. |
| For instance, such a reasoning might determine whether some source element is reachable or whether a variable is used or not. |
| To simplify this, quantors are provided. |
| Since branch walkers originating from a single activation point can fork, the reasoning has to include all these forked branch walkers. |
| Hence, graph explorers are instantiated using a quantor which can be either <emphasis>For All</emphasis>, <emphasis>At Least One</emphasis> OR <emphasis>None</emphasis> that refers to all branches. |
| After all branch walkers of an explorer are terminated, the explorer is regarded as either passed or failed. |
| Paths also can be aborted manually using the methods <literal>pass()</literal> or <literal>fail()</literal>. |
| When <emphasis>pass</emphasis> or <emphasis>fail</emphasis> are used, the graph explorer might be terminated in the following cases:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>If the quantor of the graph explorer is <emphasis>For All</emphasis>, and <literal>fail()</literal> is called on a branch walker.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>If the quantor of the graph explorer is <emphasis>At Least One</emphasis>, and <literal>pass()</literal> is called on a branch walker.</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>Additionally, a graph explorer can be aborted manually by canceling all its branches.</simpara> |
| </section> |
| </section> |
| <section xml:id="_control-flow-analyses"> |
| <title>Control flow analyses</title> |
| <section xml:id="_dead-code-analysis"> |
| <title>Dead code analysis</title> |
| <simpara>The dead code analysis uses the graph visitor in all four modes and collects all visited CF elements. |
| The collected CF elements are saved separately for every mode. |
| After the graph visitor is terminated, the unreachable CF elements are computed like follows:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>CF elements, that are collected during forward and catch block mode are reachable.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>CF elements, that are collected during islands mode are unreachable.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>CF elements, that are <emphasis>only</emphasis> collected during backward mode, are also unreachable.</simpara> |
| </listitem> |
| </itemizedlist> |
| <simpara>In a later step, the unreachable elements are merged into unreachable text regions that are used for error markers.</simpara> |
| </section> |
| </section> |
| </section> |
| <section xml:id="sec:dataflow" role="language-n4js"> |
| <title>Dataflow</title> |
| <simpara></simpara> |
| <sidebar> |
| <simpara><link xl:href="https://github.com/eclipse/n4js/issues/331"><inlinemediaobject> |
| <imageobject> |
| <imagedata fileref="images/issue.svg"/> |
| </imageobject> |
| <textobject><phrase></phrase></textobject> |
| </inlinemediaobject> #331</link> |
| <link xl:href="https://github.com/eclipse/n4js/issues/464"><inlinemediaobject> |
| <imageobject> |
| <imagedata fileref="images/issue.svg"/> |
| </imageobject> |
| <textobject><phrase></phrase></textobject> |
| </inlinemediaobject> #464</link></simpara> |
| </sidebar> |
| <section xml:id="_dataflow-graph"> |
| <title>Dataflow graph</title> |
| <simpara>The data flow graph provides means to reason about <emphasis>symbols</emphasis>, <emphasis>effects</emphasis>, <emphasis>data flow</emphasis>, <emphasis>aliases</emphasis> and <emphasis>guards</emphasis> in the control flow graph. |
| The main classes of the data flow API are <literal>DataflowVisitor</literal> and <literal>Assumption</literal>.</simpara> |
| <variablelist> |
| <varlistentry> |
| <term>Symbol</term> |
| <listitem> |
| <simpara>Symbols represent a program variable in the sence that it represents all AST elements, that bind to the same variable declaration (according to scoping). |
| The terms <emphasis>symbol</emphasis> and <emphasis>variable</emphasis> are used synonymously.</simpara> |
| </listitem> |
| </varlistentry> |
| <varlistentry> |
| <term>Effect</term> |
| <listitem> |
| <simpara>Effects are reads, writes and declarations of symbols. |
| For instance, a typical CF element with a write effect is an assignment such as <literal>a = null;</literal>. |
| Every effect refers to a single symbol and graph node. |
| The following effects are provided:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><emphasis>Declaration</emphasis>: is the declaration of a variable.</simpara> |
| </listitem> |
| <listitem> |
| <simpara><emphasis>Write</emphasis>: is the definition of a variable’s value, which is typically done with an assignment.</simpara> |
| </listitem> |
| <listitem> |
| <simpara><emphasis>Read</emphasis>: is the read of a variable’s value, which could happen when passing a variable as an argument to a method call.</simpara> |
| </listitem> |
| <listitem> |
| <simpara><emphasis>MethodCall</emphasis>: is the call of a property method of a variable.</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| </varlistentry> |
| </variablelist> |
| <simpara>Note that the term <emphasis>value use</emphasis> means either write or method call of a variable. |
| The term <emphasis>value definition</emphasis> means that a variable is written.</simpara> |
| <variablelist> |
| <varlistentry> |
| <term>Data flow</term> |
| <listitem> |
| <simpara>The term data flow is used for assignments of all kind. |
| For instance, the assigments <literal>a = b</literal>, <literal>a = 1</literal>, <literal>a = null</literal> or even <literal>for (let [a] of [[0],[undefined]]);</literal> are data flows. |
| The data is always flowing from the right hand side to the left hand side.</simpara> |
| </listitem> |
| </varlistentry> |
| <varlistentry> |
| <term>Alias</term> |
| <listitem> |
| <simpara>Due to data flow, other symbols can get important for an analysis. |
| For instance, the data flow <literal>a = b</literal> makes <literal>b</literal> important when reasoning about <literal>a</literal> since the value of <literal>b</literal> is assigned to <literal>a</literal>. |
| In the API is <literal>b</literal> therefore called an alias of <literal>a</literal>.</simpara> |
| </listitem> |
| </varlistentry> |
| <varlistentry> |
| <term>Guard</term> |
| <listitem> |
| <simpara>Guards are conditions that appear in e.g. <literal>it</literal>-statements. |
| For instance, a typical guard is the null-check in the following statement: <literal>if (a == null) foo();</literal>. |
| For every CF element, guards can hold either <emphasis>always</emphasis>, <emphasis>never</emphasis> or <emphasis>sometimes</emphasis>. |
| Note that the null-check-guard always holds at the method invocation <literal>foo();</literal>.</simpara> |
| </listitem> |
| </varlistentry> |
| <varlistentry> |
| <term><literal>DataflowVisitor</literal></term> |
| <listitem> |
| <simpara>The class <literal>DataflowVisitor</literal> provides means to visit all code locations where either effects happen or guards are declared. |
| For instance, when a variable is written, the callback method <literal>DataflowVisitor#visitEffect(EffectInfo effect, ControlFlowElement cfe)</literal> gets called. |
| In case a guard is declared, the callback method <literal>visitGuard(Guard guard)</literal> gets called.</simpara> |
| </listitem> |
| </varlistentry> |
| <varlistentry> |
| <term><literal>Assumption</literal></term> |
| <listitem> |
| <simpara>The class <literal>Assumption</literal> provides means to track the data flow of a specific symbol from a specific code location. |
| For instance, assumptions are used to detect whether the symbol <literal>s</literal> in the property access <literal>s.prop</literal> is or may be undefined. |
| In this example, the assumption symbol is <literal>s</literal> and its start location is the property access. |
| From there, the data flow of <literal>s</literal> is tracked in backwards traverse direction. |
| Also, (transitive) aliases of <literal>s</literal> are tracked. |
| In case a data flow that happens on <literal>s</literal> or its aliases, the callback method <literal>holdsOnDataflow(Symbol lhs, Symbol rSymbol, Expression rValue)</literal> is called. |
| For every effect that affects <literal>s</literal> or one of its aliases, the callback method <literal>holdsOnEffect(EffectInfo effect, ControlFlowElement container)</literal> is called. |
| And finally, for all guards that hold always/never at the start location regarding symbol <literal>s</literal>, the callback method <literal>holdsOnGuards(Multimap<GuardType, Guard> neverHolding, Multimap<GuardType, Guard> alwaysHolding)</literal> is called.</simpara> |
| </listitem> |
| </varlistentry> |
| </variablelist> |
| </section> |
| <section xml:id="_dataflow-analyses"> |
| <title>Dataflow analyses</title> |
| <section xml:id="_def-def-def-nothing-analysis"> |
| <title>Def→Def / Def→Nothing analysis</title> |
| <simpara>A Def→Def analysis finds all defintions of a variable that are always a predecessor of another definition. |
| Its result is a set of all obsolete definition sites.</simpara> |
| <simpara>A Def→!Use analysis finds all definitions of a variable that are not followed by either a read or a method call. |
| These definition are therefore obsolete and can be removed.</simpara> |
| <simpara>Both of these analyses are performed in traverse direction <emphasis>Forward</emphasis>.</simpara> |
| </section> |
| <section xml:id="_def-use-decl-analysis"> |
| <title>Def|Use←Decl analysis</title> |
| <simpara>A Def|Use←Decl analysis finds all preceding <emphasis>def</emphasis> or <emphasis>use</emphasis> sites of a declarations of a specific variable. |
| The paths might contain other <emphasis>defs</emphasis> or <emphasis>uses</emphasis> of the same variable. |
| In case such paths exists, the variable is used before it is declared. |
| This analysis is done in traverse direction <emphasis>Backward</emphasis>.</simpara> |
| <figure xml:id="img:usedBeforeDeclaredAnalysis"> |
| <title>Finding use or def sites can be done using the graph visitor in traverse direction <emphasis>Backward</emphasis>.</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/30_flowgraphs/images/usedBeforeDeclaredAnalysis.png"/> |
| </imageobject> |
| <textobject><phrase>usedBeforeDeclaredAnalysis</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara>In the <link linkend="img:usedBeforeDeclaredAnalysis">above figure</link> a graph visitor would visit all CF elements. |
| When it visits the declaration in line 8 (<literal>let w</literal>), it will activate a graph explorer (star 1 in the figure) for variable <literal>w</literal>. |
| Now, the first branch walker <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>b</mi><mn>1</mn></msub></math> is created and walks the control in backward traverse direction. |
| When <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>b</mi><mn>1</mn></msub></math> encounters the exit node of the <literal>if</literal>-statement, it will create two forked branches <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>b</mi><mn>2</mn></msub></math> and <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>b</mi><mn>3</mn></msub></math>. |
| Now, <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>b</mi><mn>2</mn></msub></math> enters then the branch of the <literal>if</literal>-statement (star 2), while <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>b</mi><mn>3</mn></msub></math> traverses directly to the condition of the <literal>if</literal>-statement. |
| Next, <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>b</mi><mn>2</mn></msub></math> visits the def site of variable <literal>w</literal> (star 3). |
| This means, that there exist a def site of <literal>w</literal> before <literal>w</literal> was declared and hence, an error should be shown. |
| Since there could exist more cases like this, neither the branch walker nor the graph explorer are terminated. |
| When reaching star 4, the two branch walkers <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>b</mi><mn>2</mn></msub></math> and <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>b</mi><mn>3</mn></msub></math> are joined and the follow-up branch walker <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>b</mi><mn>4</mn></msub></math> is created. |
| At star 5, the end the CF container is reached and the <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>b</mi><mn>4</mn></msub></math> will be terminated. |
| After all branch walkers are terminated, the graph explorer for the declaration site of variable <literal>w</literal> is evaluated: |
| All use or def sites, that were reachable should be marked with an error saying that the declaration has to be located before the use of a variable.</simpara> |
| <simpara>Note this analysis is currently implemented as a control flow analysis since it does not rely on guards, aliases. |
| Also, it only relies on local variables and hence does not need the symbols that are provided by the data flow API.</simpara> |
| </section> |
| </section> |
| </section> |
| </chapter> |
| <chapter xml:id="sec:publish-npms-to-public"> |
| <title>Publish npms</title> |
| <simpara>We publish npms located in the folder <literal>n4js-libs</literal> to the <link xl:href="registry.npmjs.org">public npm registry</link>. Specifically, the following npms are published:</simpara> |
| <variablelist> |
| <varlistentry> |
| <term>Command line tools</term> |
| <listitem> |
| <itemizedlist> |
| <listitem> |
| <simpara>n4js-cli</simpara> |
| </listitem> |
| <listitem> |
| <simpara>n4js-mangelhaft-cli</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| </varlistentry> |
| <varlistentry> |
| <term>Runtime definition files</term> |
| <listitem> |
| <itemizedlist> |
| <listitem> |
| <simpara>n4js-runtime-ecma402</simpara> |
| </listitem> |
| <listitem> |
| <simpara>n4js-runtime-es2015</simpara> |
| </listitem> |
| <listitem> |
| <simpara>n4js-runtime-esnext</simpara> |
| </listitem> |
| <listitem> |
| <simpara>n4js-runtime-fetch</simpara> |
| </listitem> |
| <listitem> |
| <simpara>n4js-runtime-html5</simpara> |
| </listitem> |
| <listitem> |
| <simpara>n4js-runtime-node</simpara> |
| </listitem> |
| <listitem> |
| <simpara>n4js-runtime-v8</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| </varlistentry> |
| <varlistentry> |
| <term>Mangelhaft</term> |
| <listitem> |
| <itemizedlist> |
| <listitem> |
| <simpara>org.eclipse.n4js.mangelhaft</simpara> |
| </listitem> |
| <listitem> |
| <simpara>org.eclipse.n4js.mangelhaft.assert</simpara> |
| </listitem> |
| <listitem> |
| <simpara>org.eclipse.n4js.mangelhaft.reporter.console</simpara> |
| </listitem> |
| <listitem> |
| <simpara>org.eclipse.n4js.mangelhaft.reporter.ide</simpara> |
| </listitem> |
| <listitem> |
| <simpara>org.eclipse.n4js.mangelhaft.reporter.xunit</simpara> |
| </listitem> |
| <listitem> |
| <simpara>org.eclipse.n4js.mangelhaft.runner.ide</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| </varlistentry> |
| </variablelist> |
| <simpara>In order to make sure that the npms work correctly with the <literal>n4js</literal> product, we need to integration test the interplay between the n4js products and the npms. Right now, we only focus on the interplay between the <literal>n4js</literal> headless compiler and npms. For integration tests, we publish the npms to a local npm registry which is provided by <link xl:href="https://www.verdaccio.org/docs/en/docker.html">verdaccio docker image</link> before executing the tests. When all integration tests are executed, we stop the local npm registry.</simpara> |
| <section xml:id="sec:publish-npms-n4js-maven"> |
| <title>Publish n4js-libs to during maven build</title> |
| <warning> |
| <simpara>The NPMs are currently published using NumberFour’s internal build infrastructure in combination with extended integration tests. This needs to be changed in the future!</simpara> |
| </warning> |
| <simpara>This section describes how integration tests can use local npm registry during the test. |
| All integration tests that require a local npm registry should be placed in the bundle <literal>org.eclipse.n4js.hlc.integrationtests</literal></simpara> |
| <figure role="center"> |
| <title>Maven phases</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/40_publish_npms/images/publish_npm_in_mvn_workflow.svg"/> |
| </imageobject> |
| <textobject><phrase>publish npm in mvn workflow</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara>During the maven build, three projects <literal>org.eclipse.n4js.external.libraries.update</literal>, <literal>org.eclipse.n4js.product.build</literal> and <literal>org.eclipse.n4js.hlc.integrations</literal> are built by maven in that order.</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>During the phase <emphasis>process-classes</emphasis> of the <literal>org.eclipse.n4js.external.libraries.update</literal> build, the UpdateShippedCode MWE2 workflow is triggerd to compile n4js code of npms in n4js-lib. The phase <emphasis>process-classes</emphasis> was chosen because it must happen after the <literal>org.eclipse.n4js.external.libraries.update</literal> bundle has been compiled.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>During the <emphasis>verify</emphasis> phase of the <literal>org.eclipse.n4js.product.build</literal> build, the Maven <literal>exec-maven</literal> plugin calls the script <literal>n4js/releng/utils/scripts/publish-n4js-libs.sh</literal> to publish the npms in the n4js-lib to the <emphasis>staging npm registry</emphasis>. The URL of this staging npm registry must be configured before triggering maven build via the environment variable <literal>NPM_STAGING_REGISTRY</literal>. Note that the staging npm registry lives beyond the life of <literal>n4js-inhouse</literal> 's maven build and holds npms that are needed by the integration tests in the n4js-extended’s build.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>During the <emphasis>pre-integration-test</emphasis> phase of the <literal>org.eclipse.n4js.hlc.integrationtests</literal> bundle, the Maven <literal>antrun</literal> plugin starts a <emphasis>verdaccio</emphasis> docker container local npm registry at <literal><link xl:href="http://localhost:4873">http://localhost:4873</link></literal> via docker. Also in the very same phase, the Maven plugin <literal>exec-maven</literal> calls the script <literal>n4js/releng/utils/scripts/publish-n4js-libs.sh</literal> to publish the npms in the n4js-lib folder to the local registry <literal><link xl:href="http://localhost:4873">http://localhost:4873</link></literal>. The list of published npms is identical to that list above. Note that the npms are published with the <literal>dist-tag</literal> <emphasis>test</emphasis>.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>During the <emphasis>integration-test</emphasis> phase of <literal>org.eclipse.n4js.hlc.integrationtests</literal>, the Maven <literal>failsafe</literal> plugin executes the integration tests. Here, the integration tests can pull the required npms from the local registry populated during the <emphasis>pre-integration-test</emphasis> above.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>In the <emphasis>pre-integration-test</emphasis> phase of <literal>org.eclipse.n4js.hlc.integrationtests</literal>, the Maven <literal>antrun</literal> plugins removes the <emphasis>verdaccio</emphasis> docker container.</simpara> |
| </listitem> |
| </itemizedlist> |
| </section> |
| </chapter> |
| <appendix xml:id="sec:Hints"> |
| <title>Hints</title> |
| <simpara>In this chapter, some tips and tricks regarding Eclipse, Xtend and Maven should be collected.</simpara> |
| <section xml:id="sec:XtextInjection"> |
| <title>Xtext Injection</title> |
| <simpara><xref linkend="fig:cd_XtextInjectors"/> shows different injectors used by Xtext |
| and their relation to the injector of a custom language created with Xtext |
| (in this example N4JS).</simpara> |
| <figure xml:id="fig:XtextInjectors" role="center"> |
| <title>Xtext injectors and custom DSL injector</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/a02_hints/images/cd_XtextInjectors.svg"/> |
| </imageobject> |
| <textobject><phrase>cd XtextInjectors</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara><emphasis role="strong">Injectors creation:</emphasis></simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>create 'SharedInjector'</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>create shared singletons</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <simpara>create (lazily) custom language injector</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>take singletons from shared injector</simpara> |
| </listitem> |
| <listitem> |
| <simpara>add bindings from 'SharedModule'</simpara> |
| </listitem> |
| <listitem> |
| <simpara>create own singletons</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| </itemizedlist> |
| <simpara>Normally one injector is bound to one language. |
| 'ContributingModule' allows custom languages to contribute bindings to the |
| shared state, effectively cross project boundaries.</simpara> |
| <simpara>It must be noted that in case of N4JS tools there are multiple languages |
| contributing / extending Xtext injector, which can be seen in figure |
| <xref linkend="fig:cd_customInjectors"/></simpara> |
| <figure xml:id="fig:cd_customInjectors" role="center"> |
| <title>Xtext injectors and custom DSL injector</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/a02_hints/images/cd_customInjectors.svg"/> |
| </imageobject> |
| <textobject><phrase>cd customInjectors</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <section xml:id="sec:DI_MultipleInjectors_Singletons"> |
| <title>Multiple Injectors and Singletons</title> |
| <simpara>Every injector creates its 'ObjectGraph'. Having multiple Injectors in |
| the system leads to multiple (disconnected) object graphs. For normal instances |
| that is not an issue, but for scoped instances this causes problems. |
| Most common issue happens with '@Singleton' instances that carry state.</simpara> |
| <figure xml:id="fig:cd_SingletonDuplicate" role="center"> |
| <title>Xtext injectors and custom DSL injector</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/a02_hints/images/cd_SingletonDuplicate.svg"/> |
| </imageobject> |
| <textobject><phrase>cd SingletonDuplicate</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara><xref linkend="fig:cd_customInjectors"/> shows situation in which both 'ChildInjector' |
| and 'N4JSInjector' have their bindings for 'N4JSEclipseCore'. As a result those |
| injectors will create their instance of core that is expected to be |
| '@Singleton'. Additionally, this will be true for all its transitive |
| dependencies.</simpara> |
| <section xml:id="sec:DI_avoid_duplicate_singletons"> |
| <title>Avoiding duplicate singletons</title> |
| <simpara>To avoid issue with duplicate singletons two distinct injectors should not |
| have their bindings for singletons. Developer needs to decide where to |
| define <emphasis role="strong">the only</emphasis> binding, and let one 'ObjectGraph' delegate to another.</simpara> |
| <section xml:id="sec:DI_binding_in_shared"> |
| <title>Defining binding in the shared injector</title> |
| <simpara>One approach is to define binding in the shared injector. Then in the injector |
| of the custom language to delegate to the shared contribution.</simpara> |
| <screen>/** Binds {@link IN4JSCore} */ |
| public class ContributingModule implements Module { |
| @Override |
| public void configure(Binder binder) { |
| binder.bind(IN4JSCore.class).to(IN4JSEclipseCore.class); |
| binder.bind(IN4JSEclipseCore.class).to(N4JSEclipseCore.class).in(SINGLETON); |
| } |
| } |
| |
| /** Delegates binding for {@link IN4JSCore} to the shared provider. */ |
| public class ContributingModule implements Module { |
| public Provider<? extends IN4JSCore> bindIN4JSCore() { |
| return Access.contributedProvider(N4JSEclipseCore.class); |
| } |
| }</screen> |
| <simpara>Downside of this approach is in the shared injector itself. |
| It does not allow for implicit bindings. This forces developer to declare |
| bindings for <emphasis role="strong">all transitive</emphasis> dependencies of the main binding explicitly. |
| Additionally, every custom language has to do it. These make shared injector |
| the <emphasis>GodInjector</emphasis> that contains configuration for all custom languages, |
| it is responsible for creating most objects in the system, and potentially |
| exposes types from one language to another language where it might not be |
| desired.</simpara> |
| </section> |
| <section xml:id="sec:DI_binding_in_custom"> |
| <title>Defining binding in the custom injector</title> |
| <simpara>Other approach is to define binding in the injector for a custom |
| language. Then let instances in the shared injector object graph to obtain |
| singleton instances via custom language injector (which is stored on the |
| custom language activator).</simpara> |
| <screen>/** Does not bind {@link IN4JSCore} */ |
| public class ContributingModule implements Module { |
| @Override |
| public void configure(Binder binder) { |
| // no core binding |
| } |
| } |
| |
| /** Binds {@link IN4JSCore}. */ |
| public class ContributingModule implements Module { |
| public Class<? extends IN4JSCore> bindIN4JSCore() { |
| return IN4JSEclipseCore.class; |
| } |
| } |
| |
| /** Some type used in shared injector object graph */ |
| public SomeSharedType{ |
| |
| /** Obtain {@link IN4JSCore} form {@code N4JSInjector}. */ |
| private IN4JSCore getIN4JSCore() { |
| return N4JSActivator |
| .getInstance() |
| .getInjector(ORG_ECLIPSE_N4JS_N4JS) |
| .getInstance(IN4JSCore.class); |
| } |
| }</screen> |
| <simpara>This approach also has downsides. In the 'SomeSharedType' that exists in the |
| shared injector object graph we cannot inject 'IN4JSCore' as it is not |
| known to the shared injector. Instead, we have to get the instance form the |
| 'N4JSInjector' manually. This requires developer to know whole (singleton) |
| types structure |
| defined in every custom language.</simpara> |
| </section> |
| </section> |
| </section> |
| <section xml:id="sec:DI_Hints" role="language-n4js"> |
| <title>Dependency Injection Hints</title> |
| <section xml:id="sec:DI_custom_bundle"> |
| <title>Use DI in custom bundle, use DI with extensions</title> |
| <section xml:id="sec:DI_custom_bundle_problem"> |
| <title>Problem</title> |
| <simpara>DI should be used in a custom bundle, i.e. a bundle not generated by Xtext. |
| E.g., a new handler should be provided in its plugin, and this handler requires |
| an injected instance. Example</simpara> |
| <simpara>my.dsl.bundle.ui xtext generated</simpara> |
| <simpara>my.dsl.bundle.sub.ui |
| The following class is contained in my custom plugin:</simpara> |
| <screen>class my.dsl.bundle.sub.ui.Handler { |
| @Inject SomeDSLOrXtextSpecificType obj; |
| }</screen> |
| <simpara>The question is, how can obj of type be injected at this location?</simpara> |
| </section> |
| <section xml:id="sec:DI_custom_bundle_solution"> |
| <title>Solution</title> |
| <simpara>First of all, to use DI in a type, the type instance itself must have been |
| created via DI. This requires an injector which uses the same class loader as |
| the type using the injector. This means that a new bundle needs its injector, |
| created by an IExecutableExtensionFactory using the bundles' activator (plugin) |
| singleton.</simpara> |
| <simpara>This activator can extend the generated activator of a Xtext bundle. The |
| following code can be used as a template, as long as no custom non-default |
| bindings are to be added (in this case, have a look at the generated activator |
| and override the methods configuring the injector):</simpara> |
| <screen>public class my.dsl.bundle.sub.ui.Activator extends my.dsl.bundle.ui.MyDSLActivator { |
| private static my.dsl.bundle.sub.ui.Activator INSTANCE; |
| |
| @Override |
| public void start(BundleContext context) throws Exception { |
| super.start(context); |
| INSTANCE = this; |
| } |
| |
| @Override |
| public void stop(BundleContext context) throws Exception { |
| INSTANCE = null; |
| super.stop(context); |
| } |
| |
| public static TypePopupActivator getInstance() { |
| return INSTANCE; |
| } |
| }</screen> |
| <simpara>Additionally, a custom 'AbstractGuiceAwareExecutableExtensionFactory' has to be |
| implemented. This class then uses the new activator instance (this is required |
| as bundles have their classloaders!)</simpara> |
| <screen>public class my.dsl.bundle.sub.ui.SubExecutableExtensionFactory extends AbstractGuiceAwareExecutableExtensionFactory { |
| @Override |
| protected Bundle getBundle() { |
| return my.dsl.bundle.sub.ui.Activator.getInstance().getBundle(); |
| } |
| |
| @Override |
| protected Injector getInjector() { |
| return my.dsl.bundle.sub.ui.Activator.getInstance().getInjector(MyDSLActivator.MY_LANGUAGE_GRAMMAR); |
| } |
| }</screen> |
| <simpara>Now, we can use this extension factory in the plugin.xml of the sub bundle to |
| let the handler be created via DI. E.g.</simpara> |
| <screen>"org.eclipse.ui.handlers"&gt; |
| <handler |
| class="my.dsl.bundle.sub.ui.SubExecutableExtensionFactory:my.dsl.bundle.sub.ui.Handler" |
| commandId="..."&gt; |
| handler&gt; |
| extension&gt;</screen> |
| </section> |
| </section> |
| <section xml:id="sec:Access_Other_DSL_Injector"> |
| <title>How do I get the Guice Injector of my language?</title> |
| <simpara>We have the use case to load a N4MF file inside the N4JS infrastructure to read |
| out the project description and configure the qualified names and container |
| visibility. I.e. we have to load another DSL in our current DSL infrastructure, |
| in the use case to have a Xtext resource set available to load the N4MF file. |
| Injecting the Xtext resource of the current DSL wouldn’t work as it has not the |
| N4MF injection context. So in the following the ways how to access this |
| injection context is described as extracted from |
| <link xl:href="http://koehnlein.blogspot.de/2012/11/xtext-tip-how-do-i-get-guice-injector.html">this blog post</link>.</simpara> |
| <section xml:id="sec:DSL_Injector_UI_context"> |
| <title>UI context</title> |
| <simpara>To access another DSL injector in a UI DSL project just add a dependency to the |
| UI project of the other DSL and then</simpara> |
| <screen>MyClass myClass = |
| TheOtherDSLActivator.getInstance().getInjector().get(MyClass.class)</screen> |
| </section> |
| <section xml:id="sec:DSL_Injector_Non_UI_context"> |
| <title>Non UI context but with injection context</title> |
| <screen>@Inject IResourceServiceProvider.Registry serviceProviderRegistry; |
| ... |
| MyClass myClass |
| = |
| serviceProviderRegistry.getResourceServiceProvider(URI.createFileURI(n4mfFileAbsolutePath)).get(MyClass.class)</screen> |
| </section> |
| <section xml:id="sec:DSL_Injector_Non_UI_non_injection_context"> |
| <title>Non UI context without injection context</title> |
| <screen>@Inject IResourceServiceProvider.Registry serviceProviderRegistry; |
| ... |
| MyClass |
| myClass |
| = |
| IResourceServiceProvider.Registry.INSTANCE.getResourceServiceProvider(uri).get(MyClass.class);</screen> |
| </section> |
| </section> |
| <section xml:id="sec:Cancel_Indicator"> |
| <title>How do I get cancel indicators in different contexts?</title> |
| <simpara>Several factors contribute to responsiveness in the IDE, but here we focus in |
| running jobs in the background and reacting to cancellation requests.</simpara> |
| <simpara>The Eclipse Jobs API is recommended for potentially long-running tasks (other |
| than incremental building, which has dedicated support). For example, the |
| outline view is populated by a background job, running validations on the |
| resource (and honoring cancellation requests initiated as for any job).</simpara> |
| <simpara>Cancel indicators are a Xtext abstraction while Eclipse favors progress |
| monitors, the latter including not only cancellation capability but also a |
| callback mechanism to give feedback in the UI about intermediate progress. |
| Cancel indicator can wrap a progress monitor.</simpara> |
| <simpara>Cancel indicators come in two variants, depending on the source of cancellation |
| events:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>a resource becoming stale (usually as a result of editing sources) triggers |
| cancellation. These cancel indicators can be obtained via |
| 'OutdatedStateManager', which itself is available via injection.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>cancel indicators associated to the UI, for example associated to an Eclipse |
| job. Examples:</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>for an outline view running in the background, an override of method |
| 'createRoot()' from 'DefaultOutlineTreeProvider' receives a UI-aware cancel |
| indicator;</simpara> |
| </listitem> |
| <listitem> |
| <simpara>for the transpiler, instances that carry cancel indicator are |
| 'IFileSystemAccess' and (in the future) 'IGenerator2'. To track the latter, see |
| Eclipse bug 477068.</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| </itemizedlist> |
| <simpara>In general, whenever a resource is validated cancel indicator should be checked |
| periodically. These checks are performed automatically via |
| 'MethodWrapperCancelable' before the (reflective) invocation of each validation |
| method and therefore require no manual intervention, see |
| 'AbstractMessageAdjustingN4JSValidator'. However, that doesn’t help in case a |
| single validation method ''takes too long''. To simplify those checks, utility |
| 'isCanceled()' of 'AbstractMessageAdjustingN4JSValidator' can be invoked.</simpara> |
| </section> |
| </section> |
| </section> |
| <section xml:id="sec:Eclipse"> |
| <title>Eclipse</title> |
| <section xml:id="sec:Show_Xtext_Index"> |
| <title>Show the current Xtext index</title> |
| <simpara>Press the following keyboard shortcut in the running UI: <keycap>CTRL</keycap><?asciidoc-br?> |
| <keycap>SHIFT</keycap> + <keycap>F3</keycap> (likely under Mac <keycap>CMD</keycap> + <keycap>SHIFT</keycap> + <keycap>F3</keycap>).</simpara> |
| </section> |
| <section xml:id="sec:Plugin_spy"> |
| <title>Plug-in spy</title> |
| <simpara>Not special for Xtext but very helpful do identify which class implements a UI |
| concept, for example, if you want to know which class implements the Open Model |
| Element dialog just press <keycap>CTRL</keycap> + <keycap>SHIFT</keycap> + <keycap>F3</keycap> to open that |
| dialog and afterwards press <keycap>SHIFT</keycap> + <keycap>ALT</keycap> + <keycap>F1</keycap> to show that |
| 'XtextEObjectSearchDialog' is used as implementation. Additionally, use |
| <keycap>SHIFT</keycap> + <keycap>ALT</keycap> + <keycap>F2</keycap> to spy buttons in the toolbar and |
| <keycap>SHIFT</keycap> + <keycap>ALT</keycap> + <keycap>F3</keycap> to spy the extension point name of the |
| currently active view or window.</simpara> |
| </section> |
| </section> |
| <section xml:id="sec:Maven-hints"> |
| <title>Maven</title> |
| <section xml:id="how-to-check-for-maven-mojo-updates"> |
| <title>How to check for Maven MOJO updates</title> |
| <simpara><emphasis role="strong">cd</emphasis> to the root directory and call</simpara> |
| <programlisting language="bash" linenumbering="unnumbered">mvn versions:display-plugin-updates</programlisting> |
| </section> |
| </section> |
| </appendix> |
| <appendix xml:id="_module-loading"> |
| <title>Module Loading</title> |
| <warning> |
| <simpara>This chapter is outdated and basically kept for historical reasons.</simpara> |
| </warning> |
| <section xml:id="sec:Dependency_Management"> |
| <title>Dependency Management</title> |
| <simpara>There exist several types of dependencies between modules, distinguishable by the time when the dependency is relevant. We first define these dependencies lazily to give an impression of the problem, at more rigorously later on.</simpara> |
| <simpara>Dependency needed at compile time. These type of dependency is removed by the compiler. These are basically type references used in variable or function declarations.</simpara> |
| <simpara>Runtime dependencies are to be handled at runtime in general. We distinguish two special types of runtime dependencies:</simpara> |
| <simpara>A loadtime dependency is a special runtime dependency that needs to be resolved before a module is initialized, that is, when all top-level statements of a module, containing class declarations, are executed. This usually is a types super type (e.g., super class), or a call to a function (defined in a different module) in a static initializer or module top level statement.</simpara> |
| <simpara>An execution time dependency is a non-initialization runtime dependency. That is, when a method is called (from another module), this is execution time.</simpara> |
| <simpara>Of course, before a module can be loaded, it needs to be fetched (i.e., the actual code has to be retrieved by the browser).</simpara> |
| <simpara>We can define sets containing modules which a given module depends on. Note that these sets contain each other, as shown in <link linkend="fig:euler_dependencies">Euler Dependencies</link>. However, we define disjoint sets in which a dependency to another type is only contained in one of the sets.</simpara> |
| <figure xml:id="fig:euler_dependencies" role="center"> |
| <title>Euler diagram of dependency sets</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/a10_moduleLoading/images/euler_dependencies.svg"/> |
| </imageobject> |
| <textobject><phrase>euler dependencies</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <simpara>Given a code sequence , we define the set of accessed modules in it as .</simpara> |
| <literallayout class="monospaced">describes all function calls happening in code block , i.e. . In case calls on functions , we define a function’s body code sequence as .</literallayout> |
| <simpara>The complete set of accessed modules for a particular code sequence is then defined as |
| ]</simpara> |
| <simpara>We explicitly allow a function to be excluded from being incorporated in the above algorithm by annotating it.</simpara> |
| <simpara>The set of load-time-dependencies for a module with initializer code is then defined as math:[\[\begin{aligned} |
| load-time-deps := AccessdModules( SuperClass(M) ) + AccessdModules( IC(M) ) \end{aligned}\]]</simpara> |
| </section> |
| <section xml:id="ecmascript-modules" role="language-javascript"> |
| <title>ECMAScript Modules</title> |
| <section xml:id="sec:ES5_Modules_Systems"> |
| <title>ES5 Modules Systems</title> |
| <simpara>Before ES6, Javascript had no built in support for modules. To overcome this hurdle, the two widely accepted formats have been :</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara><literal>CommonJS</literal> : Primarily aimed at synchronous module loading. The main implementation of this format is seen in <literal>Node.js</literal></simpara> |
| <programlisting language="javascript" linenumbering="unnumbered">var value = 100; |
| function inc() { |
| value++; |
| } |
| |
| module.exports = { |
| value : value, |
| inc : inc |
| }</programlisting> |
| <simpara>Import via require.</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>AMD</literal> : Primarily aimed at asynchronous module loading in browsers. The main implementation of this format is seen in <literal>RequireJS</literal>.</simpara> |
| <screen>define('myModule', ['mod1', 'mod2'], function (mod1, mod2) { |
| return { |
| myFunc: function(x, y) { |
| .. |
| } |
| }; |
| }; |
| });</screen> |
| <simpara><literal>passive</literal> format</simpara> |
| </listitem> |
| </orderedlist> |
| </section> |
| <section xml:id="sec:ES6_Modules"> |
| <title>ES6 Modules</title> |
| <simpara>The ES6 spec introduces <link xl:href="http://www.ecma-international.org/ecma-262/6.0/#sec-modules">modules</link>. ES6 modules resemble <literal>CommonJS</literal> syntax with <literal>AMD</literal> like asynchronous loading support.</simpara> |
| <simpara>Apart from the syntactic details, the highlights of ES6 modules are :</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>ES6 modules support (multiple) named exports.</simpara> |
| <screen>export const VERSION = "1.0.1"; |
| export function inc(x) { |
| return x + 1; |
| }</screen> |
| </listitem> |
| <listitem> |
| <simpara>ES6 modules support default exports.</simpara> |
| <screen>export default function (x) { |
| return x+1; |
| }</screen> |
| </listitem> |
| <listitem> |
| <simpara>As specified <link xl:href="http://www.ecma-international.org/ecma-262/6.0/#sec-createimportbinding">here</link>, ES6 modules export live immutable bindings (instead of values).</simpara> |
| <note> |
| <simpara>This behaviour is different from that of <literal>CommonJS</literal> and <literal>AMD</literal> modules where a snapshot of the value is exported.</simpara> |
| </note> |
| <simpara>An example demonstrating the behavioural difference :</simpara> |
| <screen>//-------------src.js------------ |
| var value = 100; |
| function inc() { |
| value++; |
| } |
| |
| module.exports = { |
| value : value, |
| inc : inc |
| } |
| //-------------main.js------------ |
| var src = require("./src"); //import src |
| |
| console.log(src.value); //prints 100 |
| src.inc(); |
| |
| console.log(src.value); //prints 100 <--- The value does not update. |
| |
| src.value = 65; |
| console.log(src.value); //prints 65 <--- The imported value is mutable.</screen> |
| <simpara>The same example with ES6 modules :</simpara> |
| <screen>//-------------src.js------------ |
| export var value = 100; // <--- ES6 syntax |
| export function inc() { // <--- ES6 syntax |
| value++; |
| } |
| |
| //-------------main.js------------ |
| import {value, inc} from "src" // <--- ES6 syntax |
| |
| console.log(value); //prints 100 |
| inc(); |
| |
| console.log(value); //prints 101 <--- The value is a live binding. |
| |
| value = 65; // <--- throws an Error implying the binding is immutable.</screen> |
| </listitem> |
| <listitem> |
| <simpara>ES6 modules impose a static module structure i.e. the imports and exports can be determined at compile time (statically).</simpara> |
| </listitem> |
| </orderedlist> |
| </section> |
| </section> |
| <section xml:id="sec:ECMAScript_Module_Loaders" role="language-n4js"> |
| <title>ECMAScript Module Loaders</title> |
| <simpara>For resolving module dependencies and loading modules, the JS landscape provides a few different module loaders.</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara><literal>RequireJS</literal> is the loader of choice for in browser, <literal>AMD</literal> style modules. We currently transpile our code into an AMD-style format to allow it running in both Browser and Node.js environments.</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>Node.js</literal> provides a native loader implementation for <literal>CommonJS</literal> style modules.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>For browsers (primarily), tools like <link xl:href="http://webpack.github.io/"><literal>Webpack</literal></link> and <link xl:href="http://browserify.org/"><literal>Browserify</literal></link> exist. These tools analyse the dependency graph of the entire project and then bundle up all the dependencies in a single file. <literal>Browserify</literal> only supports <literal>CommonJS</literal> modules where as <literal>Webpack</literal> works with both <literal>CommonJS</literal> & <literal>AMD</literal> style modules.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>At the time of writing this document (August 2015), there does not exist any native implementation for ES6 modules by any Javascript host environments i.e. ES6 modules are not natively supported by browsers or <literal>Node.js</literal>, as of now.</simpara> |
| </listitem> |
| </orderedlist> |
| <simpara>[fig:moduelLoader] shows an overview.</simpara> |
| <figure xml:id="fig:moduleLoader" role="center"> |
| <title>Module Loader and Transpilers, Overview</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="chapters/a10_moduleLoading/images/moduleLoader.svg"/> |
| </imageobject> |
| <textobject><phrase>moduleLoader</phrase></textobject> |
| </mediaobject> |
| </figure> |
| <section xml:id="sec:ES6_Module_Loaders"> |
| <title>ES6 Module Loaders</title> |
| <simpara>The ES6 spec started out with ES6 Module Loader details as part of the spec. However the Working Group later decided to not proceed with it. The specification for ES6 Module Loader is now a separate specification [<link linkend="WhatWGLoader">WhatWGLoader</link>].</simpara> |
| <simpara>The aim of this specification is:</simpara> |
| <blockquote> |
| <simpara>This specification describes the behavior of loading JavaScript modules from a JavaScript host environment. It also provides APIs for intercepting the module loading process and customizing loading behavior.</simpara> |
| </blockquote> |
| <simpara>The <link xl:href="https://github.com/whatwg/loader#implementation-status">Implementation status</link> of the spec states :</simpara> |
| <blockquote> |
| <simpara>It is too early to know about the Loader, first we need ES2015 modules implemented by the various engines.</simpara> |
| </blockquote> |
| </section> |
| <section xml:id="sec:Polyfills_for_ES6_Module_Loaders"> |
| <title>Polyfills for ES6 Module Loaders</title> |
| <simpara>Although there is no native support for ES6 module loading, there are a few attempts to polyfill this gap.</simpara> |
| <section xml:id="sec:es6_module_loader"> |
| <title>es6-module-loader</title> |
| <simpara>The <link xl:href="https://github.com/ModuleLoader/es6-module-loader"><literal>es6-module-loader</literal></link> project provides a polyfill for the ES6 Module Loader implementation. It dynamically loads ES6 modules in browsers and <literal>Node.js</literal> with support for loading existing and custom module formats through loader hooks.</simpara> |
| </section> |
| <section xml:id="sec:SystemJS"> |
| <title>SystemJS</title> |
| <simpara>Building upon <literal>es6-module-loader</literal>, <link xl:href="https://github.com/systemjs/systemjs"><literal>SystemJS</literal></link> supports loading ES6 modules along with <literal>AMD</literal>, <literal>CommonJS</literal> and global scripts in the browser and <literal>Node.js</literal>.</simpara> |
| <note> |
| <simpara>In order to use ES6 modules (written in ES6 syntax) the code first needs to be transpiled to ES5. For this purpose, <literal>SystemJS</literal> provides an option to use <link xl:href="https://github.com/google/traceur-compiler"><literal>Traceur</literal></link> compiler or <link xl:href="https://babeljs.io/"><literal>Babel</literal></link>.</simpara> |
| </note> |
| </section> |
| <section xml:id="sec:Demo"> |
| <title>Demo</title> |
| <simpara>A demonstration of how to how to use ES6 modules with <literal>Babel</literal> and <literal>SystemJS</literal> in <literal>Node.js</literal> as of today (August 2015).</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>Create an ES6 module as shown:</simpara> |
| <screen>export var value = 100; // <--- named export of a variable |
| export function inc() { // <--- named export of a function |
| value++; |
| }</screen> |
| </listitem> |
| <listitem> |
| <simpara>Import the bindings from the module as shown:</simpara> |
| <screen>import {value, inc} from "src" |
| |
| var importedValue = value; // <--- using the imported value |
| inc(); // <--- using the imported function</screen> |
| </listitem> |
| <listitem> |
| <simpara>Transpile these two files using <literal>Babel</literal> to ES5 with the target module format as <literal>system</literal>, as shown:</simpara> |
| <screen>$ babel <inputdir> --out-dir <outputdir> --modules system</screen> |
| </listitem> |
| <listitem> |
| <simpara>The transpiled output should be resemble the following:</simpara> |
| <screen>System.register([], function (_export) { |
| "use strict"; |
| |
| var value; |
| |
| _export("inc", inc); |
| |
| function inc() { |
| _export("value", value += 1); |
| } |
| |
| return { |
| setters: [], |
| execute: function () { |
| value = 100; |
| |
| _export("value", value); |
| } |
| }; |
| });</screen> |
| <screen>System.register(["src"], function (_export) { |
| "use strict"; |
| |
| var value, inc, importedValue; |
| return { |
| setters: [function (_src) { |
| value = _src.value; |
| inc = _src.inc; |
| }], |
| execute: function () { |
| importedValue = value; |
| inc(); |
| } |
| }; |
| });</screen> |
| </listitem> |
| <listitem> |
| <simpara>Finally run the above transpiled files, as shown:</simpara> |
| <screen>var System = require('systemjs'); // <--- Require SystemJS |
| System.transpiler = 'babel'; // <--- Configure SystemJS |
| |
| System.import('main'); // <--- Import the transpiled "main" module.</screen> |
| </listitem> |
| </orderedlist> |
| </section> |
| </section> |
| </section> |
| <section xml:id="case-study-typescript" role="language-typescript"> |
| <title>Case Study : TypeScript</title> |
| <note> |
| <simpara><anchor xml:id="sec:Case_Study___TypeScript" xreflabel="[sec:Case_Study___TypeScript]"/> This section is NOT an exhaustive introduction to Microsoft’s <literal>TypeScript</literal>, but a narrowed down analysis of certain aspects of <literal>TypeScript</literal>.</simpara> |
| </note> |
| <section xml:id="sec:ES6_Modules_Support"> |
| <title>ES6 Modules Support</title> |
| <simpara><literal>TypeScript</literal> language has recently added support for ES6 modules. From the <link xl:href="https://github.com/Microsoft/TypeScript/wiki/What%27s-new-in-TypeScript#es6-modules">wiki</link> :</simpara> |
| <blockquote> |
| <simpara><literal>TypeScript</literal> 1.5 supports <literal>ECMAScript 6</literal> (ES6) modules. ES6 modules are effectively <literal>TypeScript</literal> external modules with a new syntax: ES6 modules are separately loaded source files that possibly import other modules and provide a number of externally accessible exports. ES6 modules feature several new export and import declarations.</simpara> |
| </blockquote> |
| </section> |
| <section xml:id="sec:TypeScript_and_Module_Loading"> |
| <title>TypeScript and Module Loading</title> |
| <simpara><literal>TypeScript</literal> does not concern itself with providing a module loader. It is the responsibility of the host environment. However <literal>TypeScript</literal>’s compiler provides options to transpile the modules to different formats like <literal>AMD</literal>, <literal>CommonJS</literal>, <literal>ES6</literal> etc. It is the developer’s responsibility to choose an appropriate format and then use the modules with a correct module loader.</simpara> |
| <simpara>From the <link xl:href="https://github.com/Microsoft/TypeScript/issues/2242">wiki</link> again :</simpara> |
| <blockquote> |
| <simpara>TypeScript supports down-level compilation of external modules using the new ES6 syntax. When compiling with <literal>-t ES3</literal> or <literal>-t ES5</literal> a module format must be chosen using <literal>-m CommonJS</literal> or <literal>-m AMD</literal>. When compiling with <literal>-t ES6</literal> the module format is implicitly assumed to be <literal>ECMAScript 6</literal> and the compiler simply emits the original code with type annotations removed. When compiling down-level for <literal>CommonJS</literal> or <literal>AMD</literal>, named exports are emitted as properties on the loader supplied exports instance. This includes default exports which are emitted as assignments to exports.default.</simpara> |
| </blockquote> |
| <simpara>Consider the following module <literal>src.ts</literal> :</simpara> |
| <screen>export var value = 100; //<--- ES6 syntax |
| |
| export function inc() { //<--- ES6 syntax |
| value++; |
| }</screen> |
| <simpara>Compiling it to <literal>SystemJS</literal> format produces :</simpara> |
| <screen>System.register([], function(exports_1) { |
| var value; |
| function inc() { |
| (exports_1("value", ++value) - 1); |
| } |
| exports_1("inc", inc); |
| return { |
| setters:[], |
| execute: function() { |
| exports_1("value", value = 100); //<--- ES6 syntax |
| } |
| } |
| });</screen> |
| <simpara>Compiling it to <literal>CommonJS</literal> format produces :</simpara> |
| <screen>exports.value = 100; //<--- ES6 syntax |
| function inc() { |
| exports.value++; |
| } |
| exports.inc = inc;</screen> |
| <simpara>Compiling it to <literal>AMD</literal> format produces :</simpara> |
| <screen>define(["require", "exports"], function (require, exports) { |
| exports.value = 100; //<--- ES6 syntax |
| function inc() { |
| exports.value++; |
| } |
| exports.inc = inc; |
| });</screen> |
| <simpara>Compiling it to <literal>UMD</literal> format produces :</simpara> |
| <screen>(function (deps, factory) { |
| if (typeof module === 'object' && typeof module.exports === 'object') { |
| var v = factory(require, exports); if (v !== undefined) module.exports = v; |
| } |
| else if (typeof define === 'function' && define.amd) { |
| define(deps, factory); |
| } |
| })(["require", "exports"], function (require, exports) { |
| exports.value = 100; //<--- ES6 syntax |
| function inc() { |
| exports.value++; |
| } |
| exports.inc = inc; |
| });</screen> |
| <simpara>NOTE :</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>Visual Studio 2015 does not <link xl:href="http://visualstudio.uservoice.com/forums/121579-visual-studio/suggestions/7017377-support-for-es6-modules">support</link> ES6 modules at this time.</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>SystemJS</literal> supports <literal>TypeScript</literal> as a compiler. This implies <literal>TypeScript</literal> modules can be transpiled to be used with <literal>SystemJS</literal> as the module loader.</simpara> |
| </listitem> |
| </orderedlist> |
| </section> |
| </section> |
| <section xml:id="sec:Cyclic_Dependencies" role="language-js"> |
| <title>Cyclic Dependencies</title> |
| <simpara>To better analyse and evaluate <literal>SystemJS</literal> module loader and different module formats, let’s look at a cyclic dependency example from a (extremely simplified) stdlib task <literal>FixedPoint6</literal>. The outline for the example is :</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>Prepare 2 ES6 modules with a circular dependency.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Then transpile these modules to different module formats (e.g. <literal>AMD</literal>, & <literal>SystemJS</literal>).</simpara> |
| </listitem> |
| <listitem> |
| <simpara>With <literal>SystemJS</literal> as the module loader, execute the test for every transpiled module format.</simpara> |
| </listitem> |
| </orderedlist> |
| <section xml:id="sec:Setup"> |
| <title>Setup</title> |
| <simpara>Consider the following ES6 listings:</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>RoundingMode</simpara> |
| <screen>export default { |
| FLOOR : "FLOOR", |
| CEILING : "CEILING" |
| }</screen> |
| </listitem> |
| <listitem> |
| <simpara>MathContext</simpara> |
| <screen>import { default as FixedPoint6 } from "FixedPoint6"; |
| import { default as RoundingMode } from "RoundingMode"; |
| |
| let MathContext = class { |
| constructor(mode) { |
| this.mode = mode; |
| } |
| |
| divide(fp1, fp2) { |
| var quotient = FixedPoint6.getQuotient(fp1, fp2); |
| |
| if(this.mode === RoundingMode.CEILING) { |
| return new FixedPoint6(Math.ceil(quotient)); |
| } else if(this.mode === RoundingMode.FLOOR) { |
| return new FixedPoint6(Math.floor(quotient)); |
| } else { |
| throw new Error("Incorrect RoundingMode"); |
| } |
| } |
| } |
| |
| MathContext.FLOOR = new MathContext(RoundingMode.FLOOR); |
| MathContext.CEILING = new MathContext(RoundingMode.CEILING); |
| |
| export default MathContext;</screen> |
| </listitem> |
| <listitem> |
| <simpara>FixedPoint6</simpara> |
| <screen>import { default as MathContext } from "MathContext"; |
| |
| export default class FixedPoint6 { |
| constructor(number) { |
| this.value = number; |
| } |
| |
| static getQuotient(fp1, fp2) { |
| return fp1.value/fp2.value; |
| } |
| |
| divide(fp) { |
| return FixedPoint6.defaultContext.divide(this, fp); |
| } |
| } |
| |
| FixedPoint6.defaultContext = MathContext.FLOOR;</screen> |
| </listitem> |
| <listitem> |
| <simpara>Test</simpara> |
| <screen>import { default as FixedPoint6 } from "FixedPoint6"; |
| import { default as MathContext } from "MathContext"; |
| import { default as RoundingMode } from 'RoundingMode'; |
| |
| var fp1 = new FixedPoint6(20.5); |
| var fp2 = new FixedPoint6(10); |
| |
| var fp3 = fp1.divide(fp2); |
| console.log(fp1, fp2, fp3);</screen> |
| </listitem> |
| <listitem> |
| <simpara>Runner : This is the runner file to execute the test (after transpilation).</simpara> |
| <screen>var System = require('systemjs'); |
| System.transpiler = 'babel'; |
| |
| System.config({ |
| baseURL: './build', |
| "paths": { |
| "*": "*.js" |
| } |
| }); |
| |
| System.import('test').catch(function(e) { |
| console.log(e); |
| })</screen> |
| </listitem> |
| </orderedlist> |
| <simpara>Clearly <literal>MathContext</literal> & <literal>FixedPoint6</literal> have a circular dependency upon each other.</simpara> |
| </section> |
| <section xml:id="sec:Transpile_and_Execute" role="language-js"> |
| <title>Transpile and Execute</title> |
| <simpara>Transpile the above setup to different formats and execute the code using <literal>SystemJS</literal> module loader :</simpara> |
| <section xml:id="sec:Module_Format___AMD"> |
| <title>Module Format = AMD</title> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>Compile the previous set up using <literal>babel</literal> as the transpiler with <literal>AMD</literal> modules :</simpara> |
| <screen>babel src -w --out-dir build --modules amd</screen> |
| </listitem> |
| <listitem> |
| <simpara>Run the transpiled <literal>test.js</literal> with :</simpara> |
| <screen>node runner.js</screen> |
| </listitem> |
| <listitem> |
| <simpara>The execution would fail with an error like the following :</simpara> |
| <screen>Error: _FixedPoint62.default.getQuotient is not a function</screen> |
| </listitem> |
| </orderedlist> |
| </section> |
| <section xml:id="sec:Module_Format___CommonJS"> |
| <title>Module Format = CommonJS</title> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>Compile the previous set up using <literal>babel</literal> as the transpiler with <literal>CommonJS</literal> modules :</simpara> |
| <screen>babel src -w --out-dir build --modules common</screen> |
| </listitem> |
| <listitem> |
| <simpara>Run the transpiled <literal>test.js</literal> with :</simpara> |
| <screen>node runner.js</screen> |
| </listitem> |
| <listitem> |
| <simpara>The execution is successful and logs the following results :</simpara> |
| <screen>{ value: 20.5 } { value: 10 } { value: 2 }</screen> |
| </listitem> |
| </orderedlist> |
| </section> |
| <section xml:id="sec:Module_Format___SystemJS"> |
| <title>Module Format = SystemJS</title> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>Compile the previous set up using <literal>babel</literal> as the transpiler with <literal>SystemJS</literal> modules :</simpara> |
| <screen>babel src -w --out-dir build --modules system</screen> |
| </listitem> |
| <listitem> |
| <simpara>Run the transpiled <literal>test.js</literal> with :</simpara> |
| <screen>node runner.js</screen> |
| </listitem> |
| <listitem> |
| <simpara>The execution is successful and logs the following results :</simpara> |
| <screen>{ value: 20.5 } { value: 10 } { value: 2 }</screen> |
| </listitem> |
| </orderedlist> |
| </section> |
| </section> |
| <section xml:id="sec:Conclusion"> |
| <title>Conclusion</title> |
| <simpara>As observed, the test is executed successfully with <literal>CommonJS</literal> & <literal>SystemJS</literal> module formats. It however fails with <literal>AMD</literal> format (due to the circular dependency).</simpara> |
| </section> |
| </section> |
| <section xml:id="system.register-as-transpilation-target" role="language-n4js"> |
| <title>System.register as transpilation target</title> |
| <simpara>In order to integrate <literal>SystemJS</literal> as the module loader, the recommended module format is <literal>System.register</literal>. This section serves as a guide (& implementation hint) to transpile N4JS modules with <literal>System.register</literal> as the module format.</simpara> |
| <section xml:id="sec:Introduction"> |
| <title>Introduction</title> |
| <simpara>This format is best explained from its <link xl:href="https://github.com/ModuleLoader/es6-module-loader/blob/master/docs/system-register.md">documentation</link> :</simpara> |
| <blockquote> |
| <simpara>System.register can be considered as a new module format designed to support the exact semantics of ES6 modules within ES5. It is a format that was developed out of collaboration and is supported as a module output in Traceur (as instantiate), Babel and TypeScript (as system). All dynamic binding and circular reference behaviors supported by ES6 modules are supported by this format. In this way it acts as a safe and comprehensive target format for the polyfill path into ES6 modules.</simpara> |
| <simpara>To run the format, a suitable loader implementation needs to be used that understands how to execute it. Currently these include SystemJS, SystemJS Self-Executing Bundles and ES6 Micro Loader. The ES6 Module Loader polyfill also uses this format internally when transpiling and executing ES6.</simpara> |
| </blockquote> |
| <simpara>The <literal>System.register</literal> format is not very well documented. However, this format is supported by all major transpilers out there i.e. <literal>BabelJS</literal>, <literal>Traceur</literal> & <literal>TypeScript</literal> transpilers. In fact, the primary resource of this documentation has been the outputs generated by these transpilers.</simpara> |
| <section xml:id="sec:External_Transpilers"> |
| <title>External Transpilers</title> |
| <simpara>In order to follow along, it will be best to try out different ES6 syntax being transpiled to <literal>System.register</literal> format by these transpilers.</simpara> |
| <simpara>The following instructions will be useful :</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>Transpile with Traceur</simpara> |
| <programlisting language="bash" linenumbering="unnumbered">traceur --dir <SOURCE_DIR> <OUTPUT_DIR> --experimental --modules=instantiate</programlisting> |
| </listitem> |
| <listitem> |
| <simpara>Transpile with Babel</simpara> |
| <programlisting language="bash" linenumbering="unnumbered">babel <SOURCE_DIR> --out-dir <OUTPUT_DIR> --modules system</programlisting> |
| </listitem> |
| <listitem> |
| <simpara>Transpile with TypeScript</simpara> |
| <simpara>Create a file by the name of <literal>tsconfig.json</literal> in the project folder, with the following contents :</simpara> |
| <programlisting language="json" linenumbering="unnumbered">{ |
| "compilerOptions": { |
| "module": "system", |
| "target": "ES5", |
| "outDir": <OUTPUT_DIR>, |
| "rootDir": <SOURCE_DIR> |
| } |
| }</programlisting> |
| <simpara>Then transpile with :</simpara> |
| <programlisting language="javascript" linenumbering="unnumbered">tsc</programlisting> |
| </listitem> |
| </orderedlist> |
| </section> |
| <section xml:id="sec:Example_of_a_System_register_module"> |
| <title>Example of a System.register module</title> |
| <simpara>For the following ES6 code :</simpara> |
| <programlisting language="javascript" linenumbering="unnumbered">import { p as q } from './dep'; |
| |
| var s = 'local'; |
| |
| export function func() { |
| return q; |
| } |
| |
| export class C {}</programlisting> |
| <simpara>The <literal>Babel</literal> transpiler generates the following code (w/ <literal>System.register</literal> format):</simpara> |
| <programlisting language="javascript" linenumbering="unnumbered">System.register(['./dep'], function (_export) { |
| 'use strict'; |
| |
| var q, s, C; |
| |
| _export('func', func); |
| |
| function _classCallCheck(instance, Constructor) { .. } |
| |
| function func() { |
| return q; |
| } |
| |
| return { |
| setters: [function (_dep) { |
| q = _dep.p; |
| }], |
| execute: function () { |
| s = 'local'; |
| |
| C = function C() { |
| _classCallCheck(this, C); |
| }; |
| |
| _export('C', C); |
| } |
| }; |
| });</programlisting> |
| </section> |
| </section> |
| <section xml:id="sec:Structure_of_a_System_register_module"> |
| <title>Structure of a System.register module</title> |
| <simpara>Broadly speaking, a <literal>System.register</literal> module has the following structure :</simpara> |
| <programlisting language="javascript" linenumbering="unnumbered">System.register(<<DEPENDENCIES ARRAY>>, function(<<exportFn>>) { |
| <<DECLARATION SCOPE>> |
| |
| return { |
| setters: <<SETTERS ARRAY>>, |
| execute: function() { |
| <<EXECUTABLES>> |
| } |
| }; |
| });</programlisting> |
| <simpara>Highlights :</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><literal>System.register(…​)</literal> is called with 2 arguments :</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><literal><<DEPENDENCIES ARRAY>></literal> : an array of dependencies (similar to <literal>AMD</literal>) extracted from the <literal>ES6 import</literal> statements.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>a function (<literal>FN</literal>) :</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara>accepts a parameter called <literal><<exportFn>></literal>. This <literal><<exportFn>></literal> (provided by <literal>SystemJS</literal>) keeps a track of all the exports of this module & will inform all the importing modules of any changes.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>contains a <literal><<DECLARATION SCOPE>></literal> where all the functions and variables (from the source code) get hoisted to.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>returns an object with 2 properties :</simpara> |
| <itemizedlist> |
| <listitem> |
| <simpara><literal>setters</literal> : The <literal><<SETTERS ARRAY>></literal> is simply an array of functions. Each of these functions represents the imported-bindings from the <literal><<DEPENDENCIES ARRAY>></literal>. The <literal><<SETTERS ARRAY>></literal> and <literal><<DEPENDENCIES ARRAY>></literal> follow the same order.</simpara> |
| </listitem> |
| <listitem> |
| <simpara><literal>execute</literal> : <literal><<EXECUTABLES>></literal> is the rest of the body of the source code.</simpara> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| </itemizedlist> |
| </section> |
| <section xml:id="_transpilation-hints"> |
| <title>Transpilation Hints</title> |
| <simpara>By observing the existing transpilers’ output, this sub-section provides insights into the process of transpiling to this format :</simpara> |
| <section xml:id="sec:Handling_Imports"> |
| <title>Handling Imports</title> |
| <simpara>The following are ES6 code snippets with some <literal>import</literal> statements :</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>Simple Import Statement</simpara> |
| <programlisting language="javascript" linenumbering="unnumbered">import {b1} from 'B';</programlisting> |
| <simpara>A valid <literal>System.register</literal> output for this snippet would look like :</simpara> |
| <screen>System.register(['B'], function (<<exportFn>>) { //(1.) |
| |
| var b1; //(2.) |
| |
| return { |
| |
| //(3.) |
| setters: [function (_B) { |
| b1 = _B.b1; //(4.) |
| }], |
| |
| execute: function () {} |
| }; |
| });</screen> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>highlights</simpara> |
| </entry> |
| <entry> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>The <literal><<DEPENDENCIES ARRAY>></literal> is just <literal>[’B’]</literal>.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>The <literal><<DECLARATION SCOPE>></literal> simply declares the imported binding <literal>v1</literal> as a variable.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>The <literal><<SETTERS ARRAY>></literal> has 1 function. This function corresponds to the single dependency (<literal>’B’</literal>) from (1.)</simpara> |
| </listitem> |
| <listitem> |
| <simpara>The setter function accepts one argument (the exported object from <literal>’B’</literal> as <literal>_B</literal> . It then sets the local binding (i.e. local variable <literal>v1</literal>) to <literal>_B.b1</literal>.</simpara> |
| </listitem> |
| </orderedlist> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Takeaway</simpara> |
| </entry> |
| <entry> |
| <itemizedlist> |
| <listitem> |
| <simpara>An <literal>import</literal> statement is broken down into <literal><<DEPENDENCIES ARRAY>></literal> & <literal><<SETTERS ARRAY>></literal>.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Whenever the value of <literal>b1</literal> inside <literal>B</literal> is changed, <literal>SystemJS</literal> will execute the corresponding <literal>setter function</literal> in this module i.e. the 1st function in this case.</simpara> |
| </listitem> |
| </itemizedlist> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </listitem> |
| <listitem> |
| <simpara>Multiple Import Statements</simpara> |
| <screen>import { a1 as a0 } from 'A'; |
| import {b1} from 'B'; |
| import { c1 as c0 } from 'C'; |
| import {b2, b3} from 'B'; |
| import {default} from 'C'; |
| import {a2} from 'A';</screen> |
| <simpara>A valid <literal>System.register</literal> output for this snippet would look like :</simpara> |
| <screen>System.register(['A', 'B', 'C'], function (<<exportFn>>) { //(1.) |
| |
| var a0, a2, b1, b2, b3, c0, default; //(2.) |
| |
| return { |
| |
| //(3.) |
| setters: [function (_A) { |
| a0 = _A.a1; //(4.1.) |
| a2 = _A.a2; //(4.2.) |
| }, function (_B) { |
| b1 = _B.b1; |
| b2 = _B.b2; |
| b3 = _B.b3; |
| }, function (_C) { |
| c0 = _C.c1; |
| default = _C['default']; |
| }], |
| |
| |
| execute: function () {} |
| }; |
| });</screen> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>highlights</simpara> |
| </entry> |
| <entry> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>The <literal><<DEPENDENCIES ARRAY>></literal> is now a unique array <literal>[’A’, ’B’, ’C’]</literal>. Note that there are no duplicates.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>The <literal><<DECLARATION SCOPE>></literal> simply declares all the imported bindings as variables.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>The <literal><<SETTERS ARRAY>></literal> now has 3 functions. These 3 functions match the ordering of the <literal><<DEPENDENCIES ARRAY>></literal>.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>The setter function accepts one argument (the exported object from the dependency) It then sets the local bindings (i.e. local variables) from the exported value of the dependency.</simpara> |
| </listitem> |
| </orderedlist> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Takeaway</simpara> |
| </entry> |
| <entry> |
| <itemizedlist> |
| <listitem> |
| <simpara>Whenever an exported value from <literal>A</literal> is changed, <literal>SystemJS</literal> will execute the first <literal>setter function</literal> in this module.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Whenever an exported value from <literal>B</literal> is changed, <literal>SystemJS</literal> will execute the second <literal>setter function</literal> in this module.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Whenever an exported value from <literal>C</literal> is changed, <literal>SystemJS</literal> will execute the third <literal>setter function</literal> in this module.</simpara> |
| </listitem> |
| </itemizedlist> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </listitem> |
| </orderedlist> |
| </section> |
| <section xml:id="sec:__exportFn__"> |
| <title><<exportFn>></title> |
| <simpara>Before moving on to handling exports, let’s focus on the SystemJS provided <literal><<exportFn>></literal>.</simpara> |
| <simpara>This function looks similar to the following :</simpara> |
| <screen>function (name, value) { //(1.) |
| module.locked = true; |
| |
| if (typeof name == 'object') { |
| for (var p in name) |
| exports[p] = name[p]; //(2.1.) |
| } |
| else { |
| exports[name] = value; //(2.2.) |
| } |
| |
| for (var i = 0, l = module.importers.length; i < l; i++) { |
| var importerModule = module.importers[i]; |
| if (!importerModule.locked) { |
| var importerIndex = indexOf.call(importerModule.dependencies, module); |
| importerModule.setters[importerIndex](exports); //(3.) |
| } |
| } |
| |
| module.locked = false; |
| return value; //(4.) |
| }</screen> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>highlights</simpara> |
| </entry> |
| <entry> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>The <literal><<exportFn>></literal> takes 2 arguments : <literal>name</literal> & <literal>value</literal>.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>It maintains an <literal>exports</literal> object with <literal>name</literal> & <literal>value</literal>.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>For every module which imports the current module, it executes the corresponding <literal>setter function</literal>.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>It returns the <literal>value</literal>.</simpara> |
| </listitem> |
| </orderedlist> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>Takeaway</simpara> |
| </entry> |
| <entry> |
| <simpara>This <literal><<exportFn>></literal> is responsible for pushing the changes from a module to every importing module thereby implementing the live binding.</simpara> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section xml:id="sec:Handling_Exports"> |
| <title>Handling Exports</title> |
| <simpara>Now let’s focus on handling <literal>export</literal> statements.</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>Simple Exports Statement</simpara> |
| <screen>export var v =1; |
| export function f(){}</screen> |
| <simpara>A valid <literal>System.register</literal> output for this snippet would look like :</simpara> |
| <screen>System.register([], function (_export) { //(1.) |
| |
| //(2.) |
| var v; |
| function f() {} |
| |
| _export("f", f); //(4.1) |
| |
| return { |
| setters: [], |
| |
| //(3.) |
| execute: function () { |
| v = 1; //(3.1.) |
| |
| _export("v", v); //(4.2.) |
| } |
| }; |
| });</screen> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>highlights</simpara> |
| </entry> |
| <entry> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>The <literal><<exportFn>></literal> is named to as <literal>_export</literal>. (This is an implementation decision by Babel.) The name should be unique to not conflict with any user-defined variable/function names.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>The <literal><<DECLARATION SCOPE>></literal> hoists the exported variable <literal>v</literal> and the function <literal>f</literal>.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>The <literal><<EXECUTABLES>></literal> zone now contains the executable code from the source module.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Initialise the variable <literal>v1</literal> with the value extracted from the source. This essentially is the executable part of the module.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>The <literal>export</literal> function expression results in a call to the <literal>_exports</literal> function as: <literal>_export(f, f)</literal></simpara> |
| </listitem> |
| <listitem> |
| <simpara>The <literal>export</literal> statement results in a call to the <literal>_export</literal> function as: <literal>_export(v, v)</literal></simpara> |
| </listitem> |
| </orderedlist> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Takeaway</simpara> |
| </entry> |
| <entry> |
| <itemizedlist> |
| <listitem> |
| <simpara>The module’s exports statements are separated from the hoistable and executable statements.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>All the exported bindings are tracked by wrapping them inside the <literal><<exportFn>></literal>.</simpara> |
| </listitem> |
| </itemizedlist> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </listitem> |
| <listitem> |
| <simpara>Tracking Exported Bindings</simpara> |
| <simpara>To maintain live bindings, <literal>SystemJS</literal> needs to track any changes to exported bindings in order to call the <literal>setter</literal> functions of importing modules. Let’s look at an example for that :</simpara> |
| <screen>export var v1 = 1; |
| export var v2 = 2; |
| export var v3 = 3; |
| export function f() {} |
| |
| v1++; //(1.) |
| ++v2; //(2.) |
| v3 += 5; //(3.) |
| f = null; //(4.)</screen> |
| <simpara><literal>Babel</literal> output for this snippet looks like :</simpara> |
| <screen>System.register([], function (_export) { |
| |
| var v1, v2, v3; |
| |
| _export("f", f); |
| |
| function f() {} |
| |
| return { |
| setters: [], |
| |
| execute: function () { |
| v1 = 1; |
| |
| _export("v1", v1); |
| |
| v2 = 2; |
| |
| _export("v2", v2); |
| |
| v3 = 3; |
| |
| _export("v3", v3); |
| |
| _export("v1", v1 += 1); //(1.) |
| _export("v2", v2 += 1); //(2.) |
| _export("v3", v3 += 5); //(3.) |
| _export("f", f = null); //(4.) |
| } |
| }; |
| });</screen> |
| <simpara><literal>Traceur</literal> output for this snippet looks like :</simpara> |
| <screen>System.register([], function($__export) { |
| |
| var v1, v2, v3; |
| function f() {} |
| |
| $__export("f", f); |
| |
| return { |
| setters: [], |
| |
| execute: function() { |
| v1 = 1; |
| $__export("v1", v1); |
| v2 = 2; |
| $__export("v2", v2); |
| v3 = 3; |
| $__export("v3", v3); |
| |
| ($__export("v1", v1 + 1), v1++); //(1.) |
| $__export("v2", ++v2); //(2.) |
| $__export("v3", v3 += 5); //(3.) |
| $__export("f", f = null); //(4.) |
| } |
| }; |
| });</screen> |
| <simpara><literal>TypeScript</literal> output for this snippet looks like :</simpara> |
| <screen>System.register([], function(exports_1) { |
| var v1, v2, v3; |
| function f() { } |
| |
| exports_1("f", f); |
| |
| return { |
| setters:[], |
| |
| execute: function() { |
| exports_1("v1", v1 = 1); |
| exports_1("v2", v2 = 2); |
| exports_1("v3", v3 = 3); |
| |
| (exports_1("v1", ++v1) - 1); //(1.) |
| exports_1("v2", ++v2); //(2.) |
| exports_1("v3", v3 += 5); //(3.) |
| f = null; //(4.) |
| } |
| } |
| });</screen> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>highlights</simpara> |
| </entry> |
| <entry> |
| <itemizedlist> |
| <listitem> |
| <simpara>The re-assignment of <literal>v1, v2, v3 and f</literal> is wrapped inside a call to the <literal><<exportFn>></literal> with the updated value.</simpara> |
| </listitem> |
| </itemizedlist> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Takeaway</simpara> |
| </entry> |
| <entry> |
| <itemizedlist> |
| <listitem> |
| <simpara>While transpiling we need to detect if any exported binding is reassigned. In that case invoke the <literal><<exportFn>></literal> immediately with the new value.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Different transpilers perform different optimization tricks, which may be worth looking at.</simpara> |
| </listitem> |
| </itemizedlist> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </listitem> |
| <listitem> |
| <simpara>Exporting a Class extending an imported Class.</simpara> |
| <simpara>Let’s look at the following class declaration :</simpara> |
| <screen>import {A} from "A"; //<-- import class A |
| |
| export class C extends A {}</screen> |
| <simpara><literal>Babel</literal> output for this snippet looks like :</simpara> |
| <screen>System.register(["A"], function (_export) { |
| |
| var A, C; |
| |
| var _get = function get(_x, _x2, _x3) { ... }; |
| |
| function _classCallCheck(instance, Constructor) { ... } |
| |
| function _inherits(subClass, superClass) { ... } |
| |
| return { |
| setters: [function (_A2) { |
| A = _A2.A; |
| }], |
| |
| execute: function () { //(1.) |
| C = (function (_A) { |
| _inherits(C, _A); |
| |
| function C() { |
| _classCallCheck(this, C); |
| |
| _get(Object.getPrototypeOf(C.prototype), "constructor", this).apply(this, arguments); |
| } |
| |
| return C; |
| })(A); |
| |
| _export("C", C); |
| } |
| }; |
| });</screen> |
| <simpara><literal>Traceur</literal> output for this snippet looks like :</simpara> |
| <screen>System.register(["A"], function($__export) { |
| var A, C; |
| |
| return { |
| setters: [function($__m) { |
| A = $__m.A; |
| }], |
| |
| execute: function() { //(1.) |
| C = $traceurRuntime.initTailRecursiveFunction(function($__super) { |
| return $traceurRuntime.call(function($__super) { |
| function C() { |
| $traceurRuntime.superConstructor(C).apply(this, arguments); |
| } |
| return $traceurRuntime.continuation($traceurRuntime.createClass, $traceurRuntime, [C, {}, {}, $__super]); |
| }, this, arguments); |
| })(A); |
| $__export("C", C); |
| } |
| }; |
| });</screen> |
| <simpara><literal>TypeScript</literal> output for this snippet looks like :</simpara> |
| <screen>System.register(["A"], function(exports_1) { |
| var __extends = function(){ ... } |
| var A_1; |
| var C; |
| |
| return { |
| setters:[ |
| function (A_1_1) { |
| A_1 = A_1_1; |
| }], |
| |
| execute: function() { //(1.) |
| C = (function (_super) { |
| __extends(C, _super); |
| function C() { |
| _super.apply(this, arguments); |
| } |
| return C; |
| })(A_1.A); |
| exports_1("C", C); |
| } |
| } |
| });</screen> |
| <informaltable tabstyle="horizontal" frame="none" colsep="0" rowsep="0"> |
| <tgroup cols="2"> |
| <colspec colwidth="15*"/> |
| <colspec colwidth="85*"/> |
| <tbody valign="top"> |
| <row> |
| <entry> |
| <simpara>highlights</simpara> |
| </entry> |
| <entry> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>Notice how the construction of class <literal>C</literal> has now been deferred to the <literal><<EXECUTABLES>></literal> zone. It is because <literal>C</literal> depends on <literal>A</literal> being imported first.</simpara> |
| </listitem> |
| </orderedlist> |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <simpara>Takeaway</simpara> |
| </entry> |
| <entry> |
| <itemizedlist> |
| <listitem> |
| <simpara>The <literal><<DECLARATION SCOPE>></literal> is for hoisting only independent entities i.e. the ones that do not depend upon any imports. Everything else is moved to the <literal><<EXECUTABLES>></literal> region.</simpara> |
| </listitem> |
| </itemizedlist> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </listitem> |
| </orderedlist> |
| </section> |
| </section> |
| <section xml:id="sec:Examples_w__Circular_Dependencies"> |
| <title>Examples w/ Circular Dependencies</title> |
| <simpara>This section focuses on circular dependencies. The goal is to see how the transpiled output looks like and if the execution is possible.</simpara> |
| <informalexample> |
| <simpara>Source files:</simpara> |
| <screen>import B from "B"; |
| |
| export default class A {} |
| A.b = new B(); //<---</screen> |
| <screen>import A from "A"; |
| |
| export default class B {} |
| B.a = new A(); //<---</screen> |
| <simpara>Transpiled Outputs (w/ Babel):<?asciidoc-br?></simpara> |
| <screen>System.register(["B"], function (_export) { |
| "use strict"; |
| |
| var B, A; |
| |
| function _classCallCheck(instance, Constructor) {...} |
| |
| return { |
| setters: [function (_B) { |
| B = _B["default"]; |
| }], |
| execute: function () { |
| A = function A() { |
| _classCallCheck(this, A); |
| }; |
| |
| _export("default", A); |
| |
| A.b = new B(); |
| } |
| }; |
| });</screen> |
| <screen>System.register(["A"], function (_export) { |
| "use strict"; |
| |
| var A, B; |
| |
| function _classCallCheck(instance, Constructor) {...} |
| |
| return { |
| setters: [function (_A) { |
| A = _A["default"]; |
| }], |
| execute: function () { |
| B = function B() { |
| _classCallCheck(this, B); |
| }; |
| |
| _export("default", B); |
| |
| B.a = new A(); |
| } |
| }; |
| });</screen> |
| <simpara>Execution Result<?asciidoc-br?></simpara> |
| <screen>var System = require('systemjs'); |
| |
| System.import('A', 'B').then(function(resp) { |
| var a = new A(); |
| var b = new B(); |
| }).catch(function(e) { |
| console.log(e); |
| });</screen> |
| <screen>Babel : [Error: undefined is not a function]</screen> |
| </informalexample> |
| <informalexample> |
| <simpara>Source files:</simpara> |
| <screen>import {B} from "B"; |
| |
| export class A extends B{}</screen> |
| <screen>import {A} from "A"; |
| |
| export class B{} |
| class C extends A{}</screen> |
| <simpara>Transpiled Outputs (w/ Babel) :<?asciidoc-br?></simpara> |
| <screen>System.register(["B"], function (_export) { |
| "use strict"; |
| |
| var B, A; |
| |
| var _get = function get(_x, _x2, _x3) { ... }; |
| |
| function _classCallCheck(instance, Constructor) { ... } |
| |
| function _inherits(subClass, superClass) {...} |
| |
| return { |
| setters: [function (_B2) { |
| B = _B2.B; |
| }], |
| execute: function () { |
| A = (function (_B) { |
| _inherits(A, _B); |
| |
| function A() { |
| _classCallCheck(this, A); |
| |
| _get(Object.getPrototypeOf(A.prototype), "constructor", this).apply(this, arguments); |
| } |
| |
| return A; |
| })(B); |
| |
| _export("A", A); |
| } |
| }; |
| });</screen> |
| <screen>System.register(["A"], function (_export) { |
| "use strict"; |
| |
| var A, B, C; |
| |
| var _get = function get(_x, _x2, _x3) { ... }; |
| |
| function _inherits(subClass, superClass) { ... } |
| |
| function _classCallCheck(instance, Constructor) { ... } |
| |
| return { |
| setters: [function (_A2) { |
| A = _A2.A; |
| }], |
| execute: function () { |
| B = function B() { |
| _classCallCheck(this, B); |
| }; |
| |
| _export("B", B); |
| |
| C = (function (_A) { |
| _inherits(C, _A); |
| |
| function C() { |
| _classCallCheck(this, C); |
| |
| _get(Object.getPrototypeOf(C.prototype), "constructor", this).apply(this, arguments); |
| } |
| |
| return C; |
| })(A); |
| } |
| }; |
| });</screen> |
| <simpara>Execution Result<?asciidoc-br?></simpara> |
| <screen>var System = require('systemjs'); |
| |
| System.import('A','B').then(function(resp) { |
| var a = new A(); |
| }).catch(function(e) { |
| console.log(e); |
| });</screen> |
| <screen>TypeScript : [Error: Cannot read property 'prototype' of undefined] |
| Babel : [Error: Super expression must either be null or a function, not undefined]</screen> |
| </informalexample> |
| <informalexample> |
| <simpara>Source files:</simpara> |
| <screen>import B from "B"; |
| |
| class A extends B {} |
| export default class X {}</screen> |
| <screen>import X from "A"; |
| |
| export default class B {} |
| class Y extends X {}</screen> |
| <simpara>Transpiled Outputs (w/ Babel):<?asciidoc-br?></simpara> |
| <screen>System.register(["B"], function (_export) { |
| "use strict"; |
| |
| var B, A, X; |
| |
| var _get = function get(_x, _x2, _x3) { ... }; |
| |
| function _classCallCheck(instance, Constructor) { ... } |
| |
| function _inherits(subClass, superClass) { ... } |
| |
| return { |
| setters: [function (_B2) { |
| B = _B2["default"]; |
| }], |
| execute: function () { |
| A = (function (_B) { |
| _inherits(A, _B); |
| |
| function A() { |
| _classCallCheck(this, A); |
| |
| _get(Object.getPrototypeOf(A.prototype), "constructor", this).apply(this, arguments); |
| } |
| |
| return A; |
| })(B); |
| |
| X = function X() { |
| _classCallCheck(this, X); |
| }; |
| |
| _export("default", X); |
| } |
| }; |
| });</screen> |
| <screen>System.register(["A"], function (_export) { |
| "use strict"; |
| |
| var X, B, Y; |
| |
| var _get = function get(_x, _x2, _x3) { ... }; |
| |
| function _inherits(subClass, superClass) { ... } |
| |
| function _classCallCheck(instance, Constructor) { ... } |
| |
| return { |
| setters: [function (_A) { |
| X = _A["default"]; |
| }], |
| execute: function () { |
| B = function B() { |
| _classCallCheck(this, B); |
| }; |
| |
| _export("default", B); |
| |
| Y = (function (_X) { |
| _inherits(Y, _X); |
| |
| function Y() { |
| _classCallCheck(this, Y); |
| |
| _get(Object.getPrototypeOf(Y.prototype), "constructor", this).apply(this, arguments); |
| } |
| |
| return Y; |
| })(X); |
| } |
| }; |
| });</screen> |
| <simpara>Execution Result<?asciidoc-br?></simpara> |
| <screen>var System = require('systemjs'); |
| |
| System.import('A').then(function(resp) { |
| var a = new A(); |
| }).catch(function(e) { |
| console.log(e); |
| });</screen> |
| <screen>TypeScript : [Error: Cannot read property 'prototype' of undefined] |
| Babel : [[Error: Super expression must either be null or a function, not undefined]]</screen> |
| </informalexample> |
| </section> |
| <section xml:id="sec:N4JS_Examples_w__Circular_Dependencies"> |
| <title>N4JS Examples w/ Circular Dependencies</title> |
| <simpara>In order to improve our precision in conversing and discussing about different kinds of circular dependencies, this section provides the most basic examples of different kinds.</simpara> |
| <section xml:id="sec:Unresolved_Cyclic_Dependencies"> |
| <title>Unresolved Cyclic Dependencies</title> |
| <simpara>Below examples demonstrate cases when cyclic dependency cannot be resolved at all and will cause runtime errors.</simpara> |
| <example> |
| <title>Circular dependency resolution 1</title> |
| <screen>import b from "B" |
| |
| export public var number a = 1; |
| export public var number a2 = b + 1;</screen> |
| <screen>import a from "A" |
| |
| export public var number b = a + 1;</screen> |
| <screen>import a2 from "A" |
| console.log(a2); //<-- should be 3. not NaN.</screen> |
| </example> |
| <example> |
| <title>Circular dependency resolution 2</title> |
| <screen>import B from "B" |
| |
| export public class A { |
| static a = B.b + 1; |
| }</screen> |
| <screen>import A from "A" |
| export public class B { |
| static b = 1; |
| } |
| export public class B2 { |
| static b2 = A.a; |
| }</screen> |
| <screen>import B2 from "B" |
| console.log(B2.b2); //should log 2</screen> |
| </example> |
| <example> |
| <title>Circular dependency resolution 3</title> |
| <screen>import B from "B" |
| export public class A { |
| B b = new B(); |
| }</screen> |
| <screen>import A from "A" |
| export public class B { |
| A a = new A(); |
| }</screen> |
| <screen>import A from "A" |
| new A(); // should not cause a runtime error.</screen> |
| </example> |
| </section> |
| <section xml:id="sec:Variables___Functions"> |
| <title>Examples with Variables & Functions</title> |
| <example> |
| <title>Circular dependency resolution 4</title> |
| <screen>import b_fun from "B" |
| |
| export public var a2 = b_fun(); |
| export public var a = 1;</screen> |
| <screen>import a from "A" |
| |
| export public function b_fun() { |
| return a + 1; |
| }</screen> |
| <screen>import a2 from "A" |
| console.log(a2); //<-- should be 2. not NaN.</screen> |
| </example> |
| </section> |
| <section xml:id="sec:Classes"> |
| <title>Examples with Classes</title> |
| <example> |
| <title>Circular dependency resolution 5</title> |
| <screen>import B from "B" |
| export public class A { |
| static a1 = 1; |
| static a2 = B.b1; |
| }</screen> |
| <screen>import A from "A" |
| export public class B { |
| static b1 = A.a1; |
| }</screen> |
| <screen>import A from "A" |
| console.log(A.a1); //should log 1. not an error.</screen> |
| </example> |
| <example> |
| <title>Circular dependency resolution 6</title> |
| <screen>import B from "B" |
| export public class A { |
| static a1 = 1; |
| static a2 = B.b1; |
| }</screen> |
| <screen>import A from "A" |
| export public class B { |
| static b1 = -1; |
| static b2 = A.a1; |
| }</screen> |
| <screen>import A from "A" |
| console.log(A.a1);//should log 1. not an error.</screen> |
| </example> |
| <example> |
| <title>Circular dependency resolution 7</title> |
| <screen>import B from "B" |
| export public class A { |
| static a = new B(); |
| }</screen> |
| <screen>import A from "A" |
| export public class B { |
| static b = new A(); |
| }</screen> |
| <screen>import A from "A" |
| new A(); //should succeed.</screen> |
| </example> |
| </section> |
| <section xml:id="sec:Examples_with_SubClassing"> |
| <title>Examples with SubClassing</title> |
| <example> |
| <title>Circular dependency resolution 8</title> |
| <screen>import B from "B" |
| export public class A {} |
| export public class C extends B {}</screen> |
| <screen>import A from "A" |
| export public class B extends A{}</screen> |
| <screen>import C from "A" |
| new C();//should succeed.</screen> |
| </example> |
| <example> |
| <title>Circular dependency resolution 9</title> |
| <screen>import B from "B" |
| export public class A {} |
| export public class C { |
| c = new B(); |
| }</screen> |
| <screen>import A from "A" |
| export public class B extends A{}</screen> |
| <screen>import C from "A" |
| new C(); //should succeed.</screen> |
| </example> |
| </section> |
| <section xml:id="sec:Miscellaneous"> |
| <title>Miscellaneous</title> |
| <example> |
| <title>Circular dependency resolution 10</title> |
| <screen>import B from "B" |
| export public class A {} |
| new B();</screen> |
| <screen>import A from "A" |
| export public class B {} |
| new A();</screen> |
| <screen>import A from "A" |
| new A() //should succeed.</screen> |
| </example> |
| <example> |
| <title>Circular dependency resolution 11</title> |
| <screen>import B from "B" |
| export public class A {} |
| B.b1;</screen> |
| <screen>import A from "A" |
| export public class B { |
| static b1; |
| } |
| new A();</screen> |
| <screen>import A from "A" |
| new A() //should succeed.</screen> |
| </example> |
| </section> |
| </section> |
| <section xml:id="_resources"> |
| <title>Resources</title> |
| <simpara><link xl:href="https://github.com/ModuleLoader/es6-module-loader/blob/master/docs/system-register.md">Wiki</link></simpara> |
| </section> |
| </section> |
| <section xml:id="sec:CommonJS_as_transpilation_target" role="language-js"> |
| <title>CommonJS as transpilation target</title> |
| <simpara>To provide better compatibility with <literal>npm</literal> eco-system, we want to transpile <literal>N4JS</literal> code to <literal>CommonJS</literal> module format.</simpara> |
| <section xml:id="_introduction-2"> |
| <title>Introduction</title> |
| <simpara>A sample <literal>CommonJS</literal> module :</simpara> |
| <screen>var lib1 = require("/lib1"); //<-- require |
| var lib2 = require("/lib2"); //<-- require |
| |
| function fn() { |
| //...something using 'lib1' & 'lib2' |
| } |
| |
| exports.usefulFn = fn; //<--exports |
| exports.uselessValue = 42; //<--exports</screen> |
| <simpara>The <link xl:href="http://www.commonjs.org/specs/modules/1.0/">CommonJS spec</link> describes the salient features of module format as (quoted verbatim) :</simpara> |
| <blockquote> |
| <simpara>Module Context</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>In a module, there is a free variable "require", that is a |
| function.</simpara> |
| <orderedlist numeration="loweralpha"> |
| <listitem> |
| <simpara>The "require" function accepts a module identifier.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>"require" returns the exported API of the foreign module.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>If there is a dependency cycle, the foreign module may |
| not have finished executing at the time it is required by one |
| of its transitive dependencies; in this case, the object |
| returned by "require" must contain at least the exports |
| that the foreign module has prepared before the call to |
| require that led to the current module’s execution.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>If the requested module cannot be returned, "require" |
| must throw an error.</simpara> |
| </listitem> |
| </orderedlist> |
| </listitem> |
| <listitem> |
| <simpara>In a module, there is a free variable called "exports", |
| that is an object that the module may add its API to as it |
| executes.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>modules must use the "exports" object as the only means |
| of exporting.</simpara> |
| </listitem> |
| </orderedlist> |
| <simpara>Module Identifiers</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>A module identifier is a String of "terms" delimited by forward |
| slashes.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>A term must be a camelCase identifier, ".", or "..".</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Module identifiers may not have file-name extensions like ".js".</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Module identifiers may be "relative" or "top-level". A module |
| identifier is "relative" if the first term is "." or "..".</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Top-level identifiers are resolved off the conceptual module |
| name space root.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Relative identifiers are resolved relative to the identifier of |
| the module in which "require" is written and called.</simpara> |
| </listitem> |
| </orderedlist> |
| </blockquote> |
| </section> |
| <section xml:id="sec:Transpilation_Hints"> |
| <title>Transpilation Hints</title> |
| <simpara>This section examines how <literal>Babel</literal> transpiles <literal>ES6</literal> modules to <literal>CommonJS</literal> format. By observing the transpiled output from <literal>Babel</literal>, we can gather insights for transpiling <literal>N4JS</literal> modules to <literal>CommonJS</literal> format.</simpara> |
| <section xml:id="sec:Import_Statements"> |
| <title>Import Statements</title> |
| <example> |
| <title>Import an entire module (for side effects only)</title> |
| <screen>import "B"; |
| console.log(B);</screen> |
| <screen>"use strict"; |
| |
| require("B"); |
| console.log(B);</screen> |
| </example> |
| <example> |
| <title>Import single member of a module</title> |
| <screen>import {b1} from "B"; |
| b1;</screen> |
| <screen>"use strict"; |
| |
| var _B = require("B"); |
| _B.b1;</screen> |
| </example> |
| <example> |
| <title>Import multiple members of a module</title> |
| <screen>import {b1, b2} from "B"; |
| b1; |
| b2;</screen> |
| <screen>"use strict"; |
| |
| var _B = require("B"); |
| |
| _B.b1; |
| _B.b2;</screen> |
| </example> |
| <example> |
| <title>Import a single member of a module w/ an alias</title> |
| <screen>import {b3 as b4} from "B"; |
| b4 + 1;</screen> |
| <screen>"use strict"; |
| |
| var _B = require("B"); |
| |
| _B.b3 + 1;</screen> |
| </example> |
| <example> |
| <title>Import multiple members of a module w/ aliases</title> |
| <screen>import {b3 as b4, b5 as b6} from "B"; |
| b4 + 1; |
| b6 + 1;</screen> |
| <screen>"use strict"; |
| |
| var _B = require("B"); |
| |
| _B.b3 + 1; |
| _B.b5 + 1;</screen> |
| </example> |
| <example> |
| <title>Import ALL the bindings of a module</title> |
| <screen>import * as B from "B"; |
| console.log(B);</screen> |
| <screen>"use strict"; |
| |
| function _interopRequireWildcard(obj) { |
| //Babel internally tracks ES6 modules using a flag "__esModule". |
| if (obj && obj.__esModule) { |
| return obj; |
| } else { |
| //Copy over all the exported members. |
| var newObj = {}; |
| if (obj != null) { |
| for (var key in obj) { |
| if (Object.prototype.hasOwnProperty.call(obj, key)) newObj[key] = obj[key]; |
| } |
| } |
| |
| //Set the "default" as the obj itself (ES6 default export) |
| newObj["default"] = obj; |
| return newObj; |
| } |
| } |
| |
| var _B = require("B"); |
| |
| var B = _interopRequireWildcard(_B); |
| |
| console.log(B);</screen> |
| </example> |
| <example> |
| <title>Import the default export of a module</title> |
| <screen>import B from "B"; |
| console.log(B);</screen> |
| <screen>"use strict"; |
| |
| //For importing a default export, |
| //Babel checks if the obj is an ES6 module or not. |
| function _interopRequireDefault(obj) { |
| return obj && obj.__esModule ? obj : { "default": obj }; |
| } |
| |
| var _B = require("B"); |
| |
| var _B2 = _interopRequireDefault(_B); |
| |
| console.log(_B2["default"]);</screen> |
| </example> |
| </section> |
| <section xml:id="sec:Export_Statements"> |
| <title>Export Statements</title> |
| <example> |
| <title>Export a member</title> |
| <screen>let a = 1; |
| export {a};</screen> |
| <screen>"use strict"; |
| |
| //Babel makes a note that this is as an ES6 module. |
| //This information is later used when this module is imported. |
| Object.defineProperty(exports, "__esModule", { |
| value: true |
| }); |
| var a = 1; |
| |
| exports.a = a;</screen> |
| </example> |
| <example> |
| <title>Export multiple members</title> |
| <screen>let a = 1; |
| let b = true; |
| |
| export {a, b};</screen> |
| <screen>"use strict"; |
| |
| Object.defineProperty(exports, "__esModule", { |
| value: true |
| }); |
| var a = 1; |
| var b = true; |
| |
| exports.a = a; |
| exports.b = b;</screen> |
| </example> |
| <example> |
| <title>Export using alias</title> |
| <screen>let a =1; |
| export {a as b};</screen> |
| <screen>"use strict"; |
| |
| Object.defineProperty(exports, "__esModule", { |
| value: true |
| }); |
| var a = 1; |
| exports.b = a;</screen> |
| </example> |
| <example> |
| <title>Multiple exports using alias</title> |
| <screen>let a = 1, b = 2; |
| export {a as A, b as B};</screen> |
| <screen>"use strict"; |
| |
| Object.defineProperty(exports, "__esModule", { |
| value: true |
| }); |
| var a = 1, |
| b = 2; |
| exports.A = a; |
| exports.B = b;</screen> |
| </example> |
| <example> |
| <title>Simple default export</title> |
| <screen>export default 42;</screen> |
| <screen>"use strict"; |
| |
| Object.defineProperty(exports, "__esModule", { |
| value: true |
| }); |
| exports["default"] = 42; //<-- default export is treated as a special named export |
| module.exports = exports["default"]; //<-- IMPORTANT</screen> |
| </example> |
| <example> |
| <title>Default export using an alias</title> |
| <screen>let x =10; |
| export {x as default};</screen> |
| <screen>"use strict"; |
| |
| Object.defineProperty(exports, "__esModule", { |
| value: true |
| }); |
| var x = 10; |
| exports["default"] = x; |
| module.exports = exports["default"];</screen> |
| </example> |
| <example> |
| <title>Default export w/ named export</title> |
| <screen>let a = 1; |
| export {a}; |
| export default 42;</screen> |
| <screen>"use strict"; |
| |
| Object.defineProperty(exports, "__esModule", { |
| value: true |
| }); |
| var a = 1; |
| exports.a = a; |
| exports["default"] = 42;</screen> |
| </example> |
| <example> |
| <title>Default export a class</title> |
| <screen>export default class A {}</screen> |
| <screen>"use strict"; |
| |
| Object.defineProperty(exports, "__esModule", { |
| value: true |
| }); |
| |
| function _classCallCheck(...) { ... } |
| |
| var A = function A() { |
| _classCallCheck(this, A); |
| }; |
| |
| exports["default"] = A; |
| module.exports = exports["default"];</screen> |
| </example> |
| <example> |
| <title>Wildcard re-export</title> |
| <screen>export * from "A"</screen> |
| <screen>"use strict"; |
| |
| Object.defineProperty(exports, "__esModule", { |
| value: true |
| }); |
| |
| function _interopExportWildcard(obj, defaults) { |
| var newObj = defaults({}, obj); |
| delete newObj["default"]; //<-- A module's default export can not be re-exported. |
| return newObj; |
| } |
| |
| function _defaults(obj, defaults) { |
| var keys = Object.getOwnPropertyNames(defaults); |
| for (var i = 0; i < keys.length; i++) { |
| var key = keys[i]; |
| var value = Object.getOwnPropertyDescriptor(defaults, key); |
| if (value && value.configurable && obj[key] === undefined) { |
| Object.defineProperty(obj, key, value); |
| } |
| } |
| return obj; |
| } |
| |
| var _A = require("A"); |
| |
| _defaults(exports, _interopExportWildcard(_A, _defaults));</screen> |
| </example> |
| <example> |
| <title>Specific member re-export</title> |
| <screen>export {a1, a2} from "A";</screen> |
| <screen>"use strict"; |
| |
| Object.defineProperty(exports, "__esModule", { |
| value: true |
| }); |
| |
| var _A = require("A"); |
| |
| Object.defineProperty(exports, "a1", { |
| enumerable: true, |
| get: function get() { |
| return _A.a1; |
| } |
| }); |
| Object.defineProperty(exports, "a2", { |
| enumerable: true, |
| get: function get() { |
| return _A.a2; |
| } |
| });</screen> |
| </example> |
| <example> |
| <title>Specific member re-export using alias</title> |
| <screen>export {a1 as A1, a2 as A2} from "A";</screen> |
| <screen>"use strict"; |
| |
| Object.defineProperty(exports, "__esModule", { |
| value: true |
| }); |
| |
| var _A = require("A"); |
| |
| Object.defineProperty(exports, "A1", { |
| enumerable: true, |
| get: function get() { |
| return _A.a1; |
| } |
| }); |
| Object.defineProperty(exports, "A2", { |
| enumerable: true, |
| get: function get() { |
| return _A.a2; |
| } |
| });</screen> |
| </example> |
| </section> |
| <section xml:id="sec:Tracking_Live_Bindings"> |
| <title>Tracking Live Bindings</title> |
| <simpara>As specified in the section about <literal>ES6 Modules</literal> (<link linkend="sec:ES6_Modules">ES6 Modules</link>), <literal>ES6 Modules</literal> export live immutable bindings. The following listings demonstrate how <literal>Babel</literal> achieves this.</simpara> |
| <example> |
| <title>Tracking Live Binding</title> |
| <screen>export var a = 1; |
| a++;</screen> |
| <screen>"use strict"; |
| |
| Object.defineProperty(exports, "__esModule", { |
| value: true |
| }); |
| var a = 1; |
| exports.a = a; |
| exports.a = a += 1; //<-- Exported value is tracked.</screen> |
| </example> |
| </section> |
| <section xml:id="sec:A_complete_example"> |
| <title>A complete example</title> |
| <simpara>The following listings present a simple but complete example of ES6 export, import and live-binding concepts. It uses 3 simple <literal>ES6 modules</literal> called <literal>A.js, B.js and Main.js</literal>. The modules are listed alongside their <literal>CommonJS</literal> versions generated by <literal>Babel</literal>.</simpara> |
| <informalexample> |
| <screen>export var a = 1; //<-- exports a number |
| |
| export function incA() { //<-- exports a function |
| a++; |
| }</screen> |
| <screen>"use strict"; |
| |
| Object.defineProperty(exports, "__esModule", { |
| value: true |
| }); |
| |
| exports.incA = incA; |
| var a = 1; |
| |
| exports.a = a; |
| |
| function incA() { |
| exports.a = a += 1; |
| }</screen> |
| <screen>import {incA} from "./A"; //<-- Imports the function from A.js |
| |
| export function incB() { //<-- Exports a function that calls the imported function from A.js |
| incA(); |
| }</screen> |
| <screen>"use strict"; |
| |
| Object.defineProperty(exports, "__esModule", { |
| value: true |
| }); |
| exports.incB = incB; |
| |
| var _A = require("./A"); |
| |
| function incB() { |
| _A.incA(); |
| }</screen> |
| <screen>import {a} from "./A"; //<-- Imports the exported number from A.js |
| import {incB} from "./B"; //<-- Imports the exported function from B.js |
| |
| console.log(a); //<-- Prints "1" |
| incB(); //<-- This will call the "incA" function of A.js |
| console.log(a); //<--Prints "2". The imported value "a" is updated.</screen> |
| <screen>"use strict"; |
| |
| var _A = require("./A"); |
| |
| var _B = require("./B"); |
| |
| console.log(_A.a); |
| _B.incB(); |
| console.log(_A.a);</screen> |
| </informalexample> |
| </section> |
| </section> |
| <section xml:id="_resources-2"> |
| <title>Resources</title> |
| <simpara><link xl:href="http://exploringjs.com/es6/ch_modules.html">Exploring ES6 by Dr. Axel Rauschmayer</link></simpara> |
| <simpara><link xl:href="http://www.commonjs.org/specs/modules/1.0/">CommonJS spec</link></simpara> |
| <simpara><link xl:href="http://benjamn.github.io/empirenode-2015/">The Importance of import and export</link></simpara> |
| </section> |
| </section> |
| </appendix> |
| <appendix xml:id="sec:License"> |
| <title>License</title> |
| <simpara>This specification and the accompanying materials is made available |
| under the terms of the Eclipse Public License v1.0 which accompanies |
| this distribution, and is available at <link xl:href="http://www.eclipse.org/legal/epl-v10.html">http://www.eclipse.org/legal/epl-v10.html</link></simpara> |
| <bridgehead xml:id="_eclipse-public-license-v-1-0" renderas="sect2">Eclipse Public License - v 1.0</bridgehead> |
| <simpara>THE ACCOMPANYING PROGRAM IS PROVIDED UNDER THE TERMS OF THIS ECLIPSE |
| PUBLIC LICENSE (<literal>AGREEMENT</literal>). ANY USE, REPRODUCTION OR DISTRIBUTION OF |
| THE PROGRAM CONSTITUTES RECIPIENT’S ACCEPTANCE OF THIS AGREEMENT.</simpara> |
| <bridgehead xml:id="_1-definitions" renderas="sect3">1. DEFINITIONS</bridgehead> |
| <variablelist> |
| <varlistentry> |
| <term><literal>Contribution</literal> means: </term> |
| <listitem> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>in the case of the initial Contributor, the initial code and |
| documentation distributed under this Agreement, and</simpara> |
| </listitem> |
| <listitem> |
| <simpara>in the case of each subsequent Contributor:</simpara> |
| <orderedlist numeration="loweralpha"> |
| <listitem> |
| <simpara>changes to the Program, and</simpara> |
| </listitem> |
| <listitem> |
| <simpara>additions to the Program;</simpara> |
| <simpara>where such changes and/or additions to the Program originate from and |
| are distributed by that particular Contributor. A Contribution |
| ’originates’ from a Contributor if it was added to the Program by such |
| Contributor itself or anyone acting on such Contributor’s behalf. |
| Contributions do not include additions to the Program which:</simpara> |
| <orderedlist numeration="lowerroman"> |
| <listitem> |
| <simpara>are separate modules of software distributed in conjunction with the Program |
| under their own license agreement, and</simpara> |
| </listitem> |
| <listitem> |
| <simpara>are not derivative works of the Program.</simpara> |
| </listitem> |
| </orderedlist> |
| </listitem> |
| </orderedlist> |
| </listitem> |
| </orderedlist> |
| </listitem> |
| </varlistentry> |
| <varlistentry> |
| <term><literal>Contributor</literal></term> |
| <listitem> |
| <simpara>means any person or entity that distributes the Program.</simpara> |
| </listitem> |
| </varlistentry> |
| <varlistentry> |
| <term><literal>Licensed Patents</literal> </term> |
| <listitem> |
| <simpara>mean patent claims licensable by a Contributor |
| which are necessarily infringed by the use or sale of its Contribution |
| alone or when combined with the Program.</simpara> |
| </listitem> |
| </varlistentry> |
| <varlistentry> |
| <term><literal>Program</literal> </term> |
| <listitem> |
| <simpara>means the Contributions distributed in accordance with this |
| Agreement.</simpara> |
| </listitem> |
| </varlistentry> |
| <varlistentry> |
| <term><literal>Recipient</literal> </term> |
| <listitem> |
| <simpara>means anyone who receives the Program under this |
| Agreement, including all Contributors.</simpara> |
| </listitem> |
| </varlistentry> |
| </variablelist> |
| <bridgehead xml:id="_2-grant-of-rights" renderas="sect3">2. GRANT OF RIGHTS</bridgehead> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>Subject to the terms of this Agreement, each Contributor hereby |
| grants Recipient a non-exclusive, worldwide, royalty-free copyright |
| license to reproduce, prepare derivative works of, publicly display, |
| publicly perform, distribute and sublicense the Contribution of such |
| Contributor, if any, and such derivative works, in source code and |
| object code form.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Subject to the terms of this Agreement, each Contributor hereby |
| grants Recipient a non-exclusive, worldwide, royalty-free patent license |
| under Licensed Patents to make, use, sell, offer to sell, import and |
| otherwise transfer the Contribution of such Contributor, if any, in |
| source code and object code form. This patent license shall apply to the |
| combination of the Contribution and the Program if, at the time the |
| Contribution is added by the Contributor, such addition of the |
| Contribution causes such combination to be covered by the Licensed |
| Patents. The patent license shall not apply to any other combinations |
| which include the Contribution. No hardware per se is licensed |
| hereunder.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Recipient understands that although each Contributor grants the |
| licenses to its Contributions set forth herein, no assurances are |
| provided by any Contributor that the Program does not infringe the |
| patent or other intellectual property rights of any other entity. Each |
| Contributor disclaims any liability to Recipient for claims brought by |
| any other entity based on infringement of intellectual property rights |
| or otherwise. As a condition to exercising the rights and licenses |
| granted hereunder, each Recipient hereby assumes sole responsibility to |
| secure any other intellectual property rights needed, if any. For |
| example, if a third party patent license is required to allow Recipient |
| to distribute the Program, it is Recipient’s responsibility to acquire |
| that license before distributing the Program.</simpara> |
| </listitem> |
| <listitem> |
| <simpara>Each Contributor represents that to its knowledge it has sufficient |
| copyright rights in its Contribution, if any, to grant the copyright |
| license set forth in this Agreement.</simpara> |
| </listitem> |
| </orderedlist> |
| <bridgehead xml:id="_3-requirements" renderas="sect3">3. REQUIREMENTS</bridgehead> |
| <simpara>A Contributor may choose to distribute the Program in object code form |
| under its own license agreement, provided that:</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>it complies with the terms and conditions of this Agreement; and</simpara> |
| </listitem> |
| <listitem> |
| <simpara>its license agreement:</simpara> |
| <orderedlist numeration="loweralpha"> |
| <listitem> |
| <simpara>effectively disclaims on behalf of all Contributors all warranties |
| and conditions, express and implied, including warranties or conditions |
| of title and non-infringement, and implied warranties or conditions of |
| merchantability and fitness for a particular purpose;</simpara> |
| </listitem> |
| <listitem> |
| <simpara>effectively excludes on behalf of all Contributors all liability for |
| damages, including direct, indirect, special, incidental and |
| consequential damages, such as lost profits;</simpara> |
| </listitem> |
| <listitem> |
| <simpara>states that any provisions which differ from this Agreement are |
| offered by that Contributor alone and not by any other party; and</simpara> |
| </listitem> |
| <listitem> |
| <simpara>states that source code for the Program is available from such |
| Contributor, and informs licensees how to obtain it in a reasonable |
| manner on or through a medium customarily used for software exchange.</simpara> |
| </listitem> |
| </orderedlist> |
| </listitem> |
| </orderedlist> |
| <simpara>When the Program is made available in source code form:</simpara> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <simpara>it must be made available under this Agreement; and</simpara> |
| </listitem> |
| <listitem> |
| <simpara>a copy of this Agreement must be included with each copy of the |
| Program.</simpara> |
| </listitem> |
| </orderedlist> |
| <simpara>Contributors may not remove or alter any copyright notices contained |
| within the Program.</simpara> |
| <simpara>Each Contributor must identify itself as the originator of its |
| Contribution, if any, in a manner that reasonably allows subsequent |
| Recipients to identify the originator of the Contribution.</simpara> |
| <bridgehead xml:id="_4-commercial-distribution" renderas="sect3">4. COMMERCIAL DISTRIBUTION</bridgehead> |
| <simpara>Commercial distributors of software may accept certain responsibilities |
| with respect to end users, business partners and the like. While this |
| license is intended to facilitate the commercial use of the Program, the |
| Contributor who includes the Program in a commercial product offering |
| should do so in a manner which does not create potential liability for |
| other Contributors. Therefore, if a Contributor includes the Program in |
| a commercial product offering, such Contributor (<literal>Commercial |
| Contributor</literal>) hereby agrees to defend and indemnify every other |
| Contributor (<literal>Indemnified Contributor</literal>) against any losses, damages |
| and costs (collectively <literal>Losses</literal>) arising from claims, lawsuits and |
| other legal actions brought by a third party against the Indemnified |
| Contributor to the extent caused by the acts or omissions of such |
| Commercial Contributor in connection with its distribution of the |
| Program in a commercial product offering. The obligations in this |
| section do not apply to any claims or Losses relating to any actual or |
| alleged intellectual property infringement. In order to qualify, an |
| Indemnified Contributor must: a) promptly notify the Commercial |
| Contributor in writing of such claim, and b) allow the Commercial |
| Contributor to control, and cooperate with the Commercial Contributor |
| in, the defense and any related settlement negotiations. The Indemnified |
| Contributor may participate in any such claim at its own expense.</simpara> |
| <simpara>For example, a Contributor might include the Program in a commercial |
| product offering, Product X. That Contributor is then a Commercial |
| Contributor. If that Commercial Contributor then makes performance |
| claims, or offers warranties related to Product X, those performance |
| claims and warranties are such Commercial Contributor’s responsibility |
| alone. Under this section, the Commercial Contributor would have to |
| defend claims against the other Contributors related to those |
| performance claims and warranties, and if a court requires any other |
| Contributor to pay any damages as a result, the Commercial Contributor |
| must pay those damages.</simpara> |
| <bridgehead xml:id="_5-no-warranty" renderas="sect3">5. NO WARRANTY</bridgehead> |
| <simpara>EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, THE PROGRAM IS PROVIDED |
| ON AN <literal>AS IS</literal> BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, |
| EITHER EXPRESS OR IMPLIED INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES |
| OR CONDITIONS OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR |
| A PARTICULAR PURPOSE. Each Recipient is solely responsible for |
| determining the appropriateness of using and distributing the Program |
| and assumes all risks associated with its exercise of rights under this |
| Agreement , including but not limited to the risks and costs of program |
| errors, compliance with applicable laws, damage to or loss of data, |
| programs or equipment, and unavailability or interruption of operations.</simpara> |
| <bridgehead xml:id="_6-disclaimer-of-liability" renderas="sect3">6. DISCLAIMER OF LIABILITY</bridgehead> |
| <simpara>EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, NEITHER RECIPIENT NOR |
| ANY CONTRIBUTORS SHALL HAVE ANY LIABILITY FOR ANY DIRECT, INDIRECT, |
| INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING |
| WITHOUT LIMITATION LOST PROFITS), HOWEVER CAUSED AND ON ANY THEORY OF |
| LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING |
| NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OR |
| DISTRIBUTION OF THE PROGRAM OR THE EXERCISE OF ANY RIGHTS GRANTED |
| HEREUNDER, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.</simpara> |
| <bridgehead xml:id="_7-general" renderas="sect3">7. GENERAL</bridgehead> |
| <simpara>If any provision of this Agreement is invalid or unenforceable under |
| applicable law, it shall not affect the validity or enforceability of |
| the remainder of the terms of this Agreement, and without further action |
| by the parties hereto, such provision shall be reformed to the minimum |
| extent necessary to make such provision valid and enforceable.</simpara> |
| <simpara>If Recipient institutes patent litigation against any entity (including |
| a cross-claim or counterclaim in a lawsuit) alleging that the Program |
| itself (excluding combinations of the Program with other software or |
| hardware) infringes such Recipient’s patent(s), then such Recipient’s |
| rights granted under Section 2(b) shall terminate as of the date such |
| litigation is filed.</simpara> |
| <simpara>All Recipient’s rights under this Agreement shall terminate if it fails |
| to comply with any of the material terms or conditions of this Agreement |
| and does not cure such failure in a reasonable period of time after |
| becoming aware of such noncompliance. If all Recipient’s rights under |
| this Agreement terminate, Recipient agrees to cease use and distribution |
| of the Program as soon as reasonably practicable. However, Recipient’s |
| obligations under this Agreement and any licenses granted by Recipient |
| relating to the Program shall continue and survive.</simpara> |
| <simpara>Everyone is permitted to copy and distribute copies of this Agreement, |
| but in order to avoid inconsistency the Agreement is copyrighted and may |
| only be modified in the following manner. The Agreement Steward reserves |
| the right to publish new versions (including revisions) of this |
| Agreement from time to time. No one other than the Agreement Steward has |
| the right to modify this Agreement. The Eclipse Foundation is the |
| initial Agreement Steward. The Eclipse Foundation may assign the |
| responsibility to serve as the Agreement Steward to a suitable separate |
| entity. Each new version of the Agreement will be given a distinguishing |
| version number. The Program (including Contributions) may always be |
| distributed subject to the version of the Agreement under which it was |
| received. In addition, after a new version of the Agreement is |
| published, Contributor may elect to distribute the Program (including |
| its Contributions) under the new version. Except as expressly stated in |
| Sections 2(a) and 2(b) above, Recipient receives no rights or licenses |
| to the intellectual property of any Contributor under this Agreement, |
| whether expressly, by implication, estoppel or otherwise. All rights in |
| the Program not expressly granted under this Agreement are reserved.</simpara> |
| <simpara>This Agreement is governed by the laws of the State of New York and the |
| intellectual property laws of the United States of America. No party to |
| this Agreement will bring a legal action under this Agreement more than |
| one year after the cause of action arose. Each party waives its rights |
| to a jury trial in any resulting litigation.</simpara> |
| </appendix> |
| <appendix xml:id="sec:Acronyms"> |
| <title>Acronyms</title> |
| <informaltable xml:id="AC" role="language-bash" frame="all" rowsep="1" colsep="1"> |
| <tgroup cols="4"> |
| <colspec colname="col_1" colwidth="12.5*"/> |
| <colspec colname="col_2" colwidth="37.5*"/> |
| <colspec colname="col_3" colwidth="12.5*"/> |
| <colspec colname="col_4" colwidth="37.5*"/> |
| <tbody> |
| <row> |
| <entry align="center" valign="top"><simpara><literal>CDep</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">Compile-Time Dependency</emphasis></simpara></entry> |
| <entry align="center" valign="top"><simpara><literal>RDep</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">Run-Time Dependency</emphasis></simpara></entry> |
| </row> |
| <row> |
| <entry align="center" valign="top"><simpara><literal>LDep</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">Load-Time Dependency</emphasis></simpara></entry> |
| <entry align="center" valign="top"><simpara><literal>IDep</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">Initialization-Time Dependency</emphasis></simpara></entry> |
| </row> |
| <row> |
| <entry align="center" valign="top"><simpara><literal>EDep</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">Execution-Time Dependency</emphasis></simpara></entry> |
| <entry align="center" valign="top"><simpara><literal>AC</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">Acceptance Criteria</emphasis></simpara></entry> |
| </row> |
| <row> |
| <entry align="center" valign="top"><simpara><literal>ANTLR</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">ANother Tool for Language Recognition</emphasis></simpara></entry> |
| <entry align="center" valign="top"><simpara><literal>API</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">Application Programming Interface</emphasis></simpara></entry> |
| </row> |
| <row> |
| <entry align="center" valign="top"><simpara><literal>AST</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">Abstract Syntax Tree</emphasis></simpara></entry> |
| <entry align="center" valign="top"><simpara><literal>ASI</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">Automatic Semicolon Insertion</emphasis></simpara></entry> |
| </row> |
| <row> |
| <entry align="center" valign="top"><simpara><literal>AST</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">Abstract Syntax Tree</emphasis></simpara></entry> |
| <entry align="center" valign="top"><simpara><literal>BNF</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">Backus-Naur Form</emphasis></simpara></entry> |
| </row> |
| <row> |
| <entry align="center" valign="top"><simpara><literal>CA</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">Content-Assist</emphasis></simpara></entry> |
| <entry align="center" valign="top"><simpara><literal>CSP</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">Constraint Satisfaction Problem</emphasis></simpara></entry> |
| </row> |
| <row> |
| <entry align="center" valign="top"><simpara><literal>CLI</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">Command Line Interface</emphasis></simpara></entry> |
| <entry align="center" valign="top"><simpara><literal>DOM</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">Document Object Model</emphasis></simpara></entry> |
| </row> |
| <row> |
| <entry align="center" valign="top"><simpara><literal>DSL</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">Domain Specific Language</emphasis></simpara></entry> |
| <entry align="center" valign="top"><simpara><literal>EBNF</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">Extended Backus-Naur Form</emphasis></simpara></entry> |
| </row> |
| <row> |
| <entry align="center" valign="top"><simpara><literal>EMF</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">Eclipse Modeling Framework</emphasis></simpara></entry> |
| <entry align="center" valign="top"><simpara><literal>EPL</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">Eclipse Public License</emphasis></simpara></entry> |
| </row> |
| <row> |
| <entry align="center" valign="top"><simpara><literal>FQN</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">Fully Qualified Name</emphasis></simpara></entry> |
| <entry align="center" valign="top"><simpara><literal>GLB</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">Greatest Lower Bound, also known as <emphasis role="strong">infimum</emphasis></emphasis></simpara></entry> |
| </row> |
| <row> |
| <entry align="center" valign="top"><simpara><literal>GPL</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">GNU General Public License</emphasis></simpara></entry> |
| <entry align="center" valign="top"><simpara><literal>IDE</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">Integrated Development Environment</emphasis></simpara></entry> |
| </row> |
| <row> |
| <entry align="center" valign="top"><simpara><literal>IDL</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">Interface Definition Language</emphasis></simpara></entry> |
| <entry align="center" valign="top"><simpara><literal>LSP</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">Liskov Substitution Principle</emphasis></simpara></entry> |
| </row> |
| <row> |
| <entry align="center" valign="top"><simpara><literal>LUB</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">Least Upper Bound, also known as <emphasis role="strong">supremum</emphasis></emphasis></simpara></entry> |
| <entry align="center" valign="top"><simpara><literal>N4JS</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">NumberFour JavaScript</emphasis></simpara></entry> |
| </row> |
| <row> |
| <entry align="center" valign="top"><simpara><literal>UI</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">User Interface</emphasis></simpara></entry> |
| <entry align="center" valign="top"><simpara><literal>UML</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">Unified Modeling Language</emphasis></simpara></entry> |
| </row> |
| <row> |
| <entry align="center" valign="top"><simpara><literal>VM</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">Virtual Machine</emphasis></simpara></entry> |
| <entry align="center" valign="top"><simpara><literal>XML</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">Extensible Markup Language</emphasis></simpara></entry> |
| </row> |
| <row> |
| <entry align="center" valign="top"><simpara><literal>XSLT</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong"><link linkend="XSL">XSL</link> Transformations</emphasis></simpara></entry> |
| <entry align="center" valign="top"><simpara><literal>XSL <anchor xml:id="XSL" xreflabel="[XSL]"/></literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">Extensible Stylesheet Language</emphasis></simpara></entry> |
| </row> |
| <row> |
| <entry align="center" valign="top"><simpara><literal>WYSIWYG</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">What You See Is What You Get</emphasis></simpara></entry> |
| <entry align="center" valign="top"><simpara><literal>WLOG</literal></simpara></entry> |
| <entry align="center" valign="top"><simpara><emphasis role="strong">without loss of generality</emphasis></simpara></entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </appendix> |
| <appendix xml:id="_bibliography-and-footnotes"> |
| <title>Bibliography and Footnotes</title> |
| <simpara><anchor xml:id="N4JSSpec" xreflabel="[N4JSSpec]"/>N4JS Project. (2018). <emphasis>N4JS Language Specification</emphasis>. Retrieved from <link xl:href="https://www.eclipse.org/n4js/spec/N4JSSpec.html">https://www.eclipse.org/n4js/spec/N4JSSpec.html</link></simpara> |
| <simpara><anchor xml:id="Xpect" xreflabel="[Xpect]"/><emphasis>Xpect, Project Website</emphasis>. Retrieved from <link xl:href="https://projects.eclipse.org/projects/modeling.xpect">https://projects.eclipse.org/projects/modeling.xpect</link></simpara> |
| <simpara><anchor xml:id="RFC8259" xreflabel="[RFC8259]"/>Bray, Tim. (2017). <emphasis>RFC 8259: The javascript object notation (json) data interchange format</emphasis>. </simpara> |
| <simpara><anchor xml:id="ECMA404" xreflabel="[ECMA404]"/>International, ECMA. (2017). <emphasis>Standard ECMA-404, The JSON Data Interchange Syntax</emphasis>. </simpara> |
| <simpara><anchor xml:id="RFC7158" xreflabel="[RFC7158]"/>Bray, Tim. (2014). <emphasis>RFC 7158: The JavaScript Object Notation ({JSON}) Data Interchange Format</emphasis>. </simpara> |
| <simpara><anchor xml:id="Ser18" xreflabel="[Ser18]"/>Seriot, Nicolas. (2018). <emphasis>Parsing JSON is a Minefield</emphasis>. Retrieved from <link xl:href="http://seriot.ch/parsing_json.php">http://seriot.ch/parsing_json.php</link></simpara> |
| <simpara><anchor xml:id="ECMA15a" xreflabel="[ECMA15a]"/>ECMA. (2015). <emphasis>ECMAScript 2015 Language Specification</emphasis>. Retrieved from <link xl:href="http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-262.pdf">http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-262.pdf</link></simpara> |
| <simpara><anchor xml:id="WhatWGLoader" xreflabel="[WhatWGLoader]"/>WhatWG. <emphasis>Loader: A Collection of Interesting Ideas</emphasis>. Retrieved from <link xl:href="http://whatwg.github.io/loader/">http://whatwg.github.io/loader/</link></simpara> |
| </appendix> |
| </book> |
