Google Git
Sign in
eclipse / gerrit / www.eclipse.org / n4js / 0f05d1ab3705f66f57f5e24fd62cf4c362fa37c8 / . / design / N4JSDesign.html
blob: 6e1e07a7ee94220a7a031752ebf44217f11ae925 [file] [log] [blame]
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<!--[if IE]><meta http-equiv="X-UA-Compatible" content="IE=edge"><![endif]-->
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="generator" content="Asciidoctor 1.5.5">
<meta name="author" content="2019-08-08 13:15:33 CEST">
<title>N4JS Design Specification</title>
<link rel="stylesheet" href="styles/spec.min.css">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.6.3/css/font-awesome.min.css">
<!-- ************* docinfo ******************************************************************* -->
<!-- ************* Favicon ************-->
<link rel="icon" href="images/favicon.ico" />
<!-- ************* Back-to-top JQuery ************* -->
<script src="https://code.jquery.com/jquery-1.12.4.js"></script>
<script src="https://code.jquery.com/ui/1.12.0/jquery-ui.js"></script>
<link href="styles/prism.min.css" rel="stylesheet" />
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/MathJax.js?config=TeX-MML-AM_CHTML">
</script>
<!-- ************* Styles ************* -->
<link rel="stylesheet" type="text/css" href="styles/n4jsspec-adoc.css">
<!-- ****************** NavBar ****************** -->
<div id="menubar">
<div class="banner">
<a href="https://www.eclipse.org/n4js/#"><img id="logo" src="images/n4js-logo.png" alt="Eclipse N4JS"></a>
</div>
<ul>
<li><a href="index.html">Index</a></li>
</ul>
</div>
<!-- ************* docinfo ******************************************************************* -->
<style>
.admonitionblock td.icon .icon-todo:before{content:"\f249";color:#f4ee42}
</style>
</head>
<body class="book toc2 toc-left">
<div id="header">
<h1>N4JS Design Specification</h1>
<div class="details">
<span id="author" class="author">2019-08-08 13:15:33 CEST</span><br>
<span id="revnumber">version 0.9</span>
</div>
<div id="toc" class="toc2">
<div id="toctitle">Table of Contents</div>
<ul class="sectlevel1">
<li><a href="#_introduction">1. Introduction</a>
<ul class="sectlevel2">
<li><a href="#notation">1.1. Notation</a></li>
<li><a href="#sec:IDE_Overview">1.2. IDE Components</a>
<ul class="sectlevel3">
<li><a href="#sec:Naming_Conventions">1.2.1. Naming Conventions</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#_eclipse-setup">2. Eclipse Setup</a>
<ul class="sectlevel2">
<li><a href="#_system-requirements">2.1. System Requirements</a></li>
<li><a href="#_contribute">2.2. Contribute</a>
<ul class="sectlevel3">
<li><a href="#_eclipse-installer">2.2.1. Eclipse Installer</a>
<ul class="sectlevel4">
<li><a href="#_changing-the-setup-script">2.2.1.1. Changing the Setup Script</a></li>
</ul>
</li>
<li><a href="#_manual-ide-configuration">2.2.2. Manual IDE Configuration</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#_release-engineering">3. Release Engineering</a>
<ul class="sectlevel2">
<li><a href="#_nightly-build-on-eclipse-infrastructure">3.1. Nightly build on Eclipse infrastructure</a></li>
<li><a href="#_build-the-n4js-ide-from-command-line">3.2. Build the N4JS IDE from command line</a>
<ul class="sectlevel3">
<li><a href="#_publish-maven-tooling-code-org-eclipse-n4js-releng-util-code">3.2.1. Publish maven-tooling <code>org.eclipse.n4js.releng.util</code></a></li>
<li><a href="#sec:test-verdaccio">3.2.2. Test Verdaccio containing n4js-libs</a></li>
<li><a href="#_generation-of-eclipse-help-for-spec-and-design-document">3.2.3. Generation of Eclipse help for spec and design document</a></li>
</ul>
</li>
<li><a href="#_updating-frameworks-and-dependencies">3.3. Updating frameworks and dependencies</a>
<ul class="sectlevel3">
<li><a href="#_update-of-eclipse-emf-xtext-etc">3.3.1. Update of Eclipse, EMF, Xtext, etc.</a></li>
<li><a href="#_update-of-the-embedded-jre">3.3.2. Update of the embedded JRE</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#_tips-and-tricks">4. Tips and Tricks</a>
<ul class="sectlevel2">
<li><a href="#_naming">4.1. Naming</a></li>
<li><a href="#_logging">4.2. Logging</a></li>
<li><a href="#_cancellation-handling">4.3. Cancellation Handling</a></li>
<li><a href="#_caching">4.4. Caching</a></li>
<li><a href="#_dependency-injection">4.5. Dependency Injection</a></li>
<li><a href="#_miscellaneous">4.6. Miscellaneous</a></li>
</ul>
</li>
<li><a href="#_parser">5. Parser</a>
<ul class="sectlevel2">
<li><a href="#sec:Parser_Overview">5.1. Overview</a></li>
<li><a href="#sec:N4JS_Parser">5.2. N4JS Parser</a></li>
<li><a href="#sec:Parser_Generation_Post_Processing">5.3. Parser Generation Post-Processing</a>
<ul class="sectlevel3">
<li><a href="#sec:Automatic_Semicolon_Insertion">5.3.1. Automatic Semicolon Insertion</a>
<ul class="sectlevel4">
<li><a href="#sec:Injected_code_in_the_Antlr_grammar_file">5.3.1.1. Injected code in the Antlr grammar file</a></li>
<li><a href="#sec:Customized_error_recovery">5.3.1.2. Customized error recovery</a></li>
</ul>
</li>
<li><a href="#sec:_No_line_terminator_allowed_here__handling">5.3.2. Async and <code>No line terminator allowed here</code> Handling</a></li>
<li><a href="#sec:Regular_Expression">5.3.3. Regular Expression</a></li>
<li><a href="#sec:Unicode">5.3.4. Unicode</a></li>
<li><a href="#sec:Literals">5.3.5. Literals</a></li>
</ul>
</li>
<li><a href="#sec:Modifiers">5.4. Modifiers</a></li>
<li><a href="#sec:Conflict_Resolutions">5.5. Conflict Resolutions</a>
<ul class="sectlevel3">
<li><a href="#sec:Reserved_Keywords_vs__Identifier_Names">5.5.1. Reserved Keywords vs. Identifier Names</a></li>
<li><a href="#sec:Operators_and_Generics">5.5.2. Operators and Generics</a></li>
</ul>
</li>
<li><a href="#sec:Content_Assist_Parser">5.6. Content-Assist Parser</a></li>
</ul>
</li>
<li><a href="#_type-system">6. Type System</a>
<ul class="sectlevel2">
<li><a href="#sec:Type_Model_and_Grammar">6.1. Type Model and Grammar</a>
<ul class="sectlevel3">
<li><a href="#sec:Type_Model_Overview">6.1.1. Type Model Overview</a></li>
<li><a href="#sec:Built_in_Types">6.1.2. Built-in and Primitive Types</a></li>
<li><a href="#sec:Type_Model_DSL">6.1.3. Type Model DSL</a></li>
</ul>
</li>
<li><a href="#sec:Type_System_Implementation">6.2. Type System Implementation</a></li>
<li><a href="#sec:Type_Inference_combined_with_AST_Traversal">6.3. Type Inference of AST</a>
<ul class="sectlevel3">
<li><a href="#sec:Type_Inference_combined_with_AST_Traversal__Background">6.3.1. Background</a></li>
<li><a href="#sec:Triggering_Type_Inference_of_AST">6.3.2. Triggering</a></li>
<li><a href="#sec:Traversal_Order_During_Type_Inference_of_AST">6.3.3. Traversal Order</a></li>
<li><a href="#sec:Cross_References_During_Type_Inference_of_AST">6.3.4. Cross-References</a></li>
<li><a href="#sec:Function_Accessor_Bodies_During_Type_Inference_of_AST">6.3.5. Function/Accessor Bodies</a></li>
<li><a href="#sec:Poly_Expressions_During_Type_Inference_of_AST">6.3.6. Poly Expressions</a></li>
<li><a href="#sec:Constraint_Solver_used_During_Type_Inference_of_AST">6.3.7. Constraint Solver</a></li>
<li><a href="#sec:Type_Guards_During_Type_Inference_of_AST">6.3.8. Type Guards</a></li>
</ul>
</li>
<li><a href="#sec:Structural_Typing">6.4. Structural Typing</a></li>
</ul>
</li>
<li><a href="#_type-index">7. Type Index</a>
<ul class="sectlevel2">
<li><a href="#sec:Type_Index_Design_Rationale">7.1. Design Rationale</a>
<ul class="sectlevel3">
<li><a href="#sec:Getting_the_Xtext_Index_Content_IResourceDescriptions">7.1.1. Getting the Xtext Index (<code>IResourceDescriptions</code>) Content</a></li>
</ul>
</li>
<li><a href="#sec:Design_Overview">7.2. Design Overview</a></li>
<li><a href="#sec:N4JS_Resource_Load_States">7.3. N4JS Resource Load States</a></li>
<li><a href="#sec:Type_Builder">7.4. Types Builder</a>
<ul class="sectlevel3">
<li><a href="#sec:Type_Inference_not_allowed_in_Type_Builder">7.4.1. Type Inference not allowed in Types Builder</a></li>
<li><a href="#sec:ComputedTypeReferences">7.4.2. Deferred Type References</a></li>
<li><a href="#sec:Use_cases_of_ComputedTypeRef">7.4.3. Use cases of DeferredTypeRef</a></li>
</ul>
</li>
<li><a href="#sec:Incremental_Builder_Overview">7.5. Incremental Builder (Overview)</a>
<ul class="sectlevel3">
<li><a href="#sec:Incremental_Builder_Overview__XtextBuilder">7.5.1. XtextBuilder</a></li>
<li><a href="#sec:Incremental_Builder_Overview__IBuilderState">7.5.2. IBuilderState</a>
<ul class="sectlevel4">
<li><a href="#copy-and-update-xtext-index">7.5.2.1. Copy and Update Xtext Index</a></li>
<li><a href="#build-state-setup-phase">7.5.2.2. Build State Setup Phase</a></li>
<li><a href="#process-queued-uris">7.5.2.3. Process Queued URIs</a></li>
<li><a href="#queueing-affected-resources">7.5.2.4. Queueing Affected Resources</a></li>
</ul>
</li>
<li><a href="#sec:Incremental_Builder_Overview__Example">7.5.3. Example</a></li>
</ul>
</li>
<li><a href="#dirty-state-handling">7.6. Dirty state handling</a>
<ul class="sectlevel3">
<li><a href="#use-case-restoring-types-from-user-data">7.6.1. Use case: Restoring types from user data</a></li>
<li><a href="#use-case-updating-the-xtext-index">7.6.2. Use case: Updating the Xtext index</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#_project-model">8. Project Model</a>
<ul class="sectlevel2">
<li><a href="#sec:Package_json">8.1. Package.json File</a></li>
<li><a href="#_accessing-project-information">8.2. Accessing Project Information</a>
<ul class="sectlevel3">
<li><a href="#sec:IN4JSCore">8.2.1. IN4JSCore</a></li>
<li><a href="#sec:N4JSModel">8.2.2. N4JSModel</a></li>
<li><a href="#sec:N4JSWorkspace">8.2.3. N4JSWorkspace</a></li>
<li><a href="#sec:N4JSProject">8.2.4. N4JSProject</a></li>
<li><a href="#sec:SourceContainer">8.2.5. SourceContainer</a></li>
<li><a href="#sec:N4JSProjectsStateHelper">8.2.6. N4JSProjectsStateHelper</a></li>
</ul>
</li>
<li><a href="#sec:Caching">8.3. Caching</a>
<ul class="sectlevel3">
<li><a href="#_caching-of-externallibraryworkspace">8.3.1. Caching of ExternalLibraryWorkspace</a></li>
<li><a href="#_caching-of-n4jsprojectsstatehelper">8.3.2. Caching of N4JSProjectsStateHelper</a></li>
</ul>
</li>
<li><a href="#sec:WildcardPathFilter">8.4. WildcardPathFilter</a></li>
<li><a href="#sec:ProjectUtils">8.5. ProjectUtils</a></li>
</ul>
</li>
<li><a href="#_binding">9. Binding</a>
<ul class="sectlevel2">
<li><a href="#sec:Binding_Design_Rationale">9.1. Design Rationale</a></li>
<li><a href="#sec:Binding_to_Members">9.2. Binding to Members</a></li>
<li><a href="#sec:Binding_Getter_Setter">9.3. Getter / Setter Binding</a></li>
<li><a href="#chap:Statics">9.4. Static Member Binding</a></li>
<li><a href="#sec:Binding_Enumeration">9.5. Enumeration Literals Binding</a></li>
<li><a href="#sec:Accessibility_of_types_and_members">9.6. Accessibility of types and members</a></li>
<li><a href="#sec:Member_Scope_Example">9.7. Member Scope Example</a></li>
<li><a href="#sec:Scoping_for_Members_of_Composed_Type_Explained">9.8. Scoping for Members of Composed Type (Union/Intersection) Example</a></li>
<li><a href="#sec:Binding_of_Structurally_References_Types">9.9. Structurally References Types</a></li>
<li><a href="#sec:Building">9.10. Building</a>
<ul class="sectlevel3">
<li><a href="#sec:Build_Phases">9.10.1. Build Phases</a></li>
<li><a href="#sec:Build_Scenarios">9.10.2. Build Scenarios</a></li>
<li><a href="#sec:Lazy_linking_problem">9.10.3. Lazy linking problem</a></li>
</ul>
</li>
<li><a href="#sec:Proxies_and_Proxy_Resolution">9.11. Proxies and Proxy Resolution (Overview)</a>
<ul class="sectlevel3">
<li><a href="#xtexts-lazy-linking-proxies">9.11.1. Xtext’s Lazy Linking Proxies</a></li>
<li><a href="#standard-emf-proxies">9.11.2. Standard EMF Proxies</a></li>
<li><a href="#_how-is-proxy-resolution-triggered">9.11.3. How is Proxy Resolution Triggered?</a></li>
<li><a href="#_when-is-proxy-resolution-allowed">9.11.4. When is Proxy Resolution Allowed?</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#_validation">10. Validation</a>
<ul class="sectlevel2">
<li><a href="#sec:validation_overview">10.1. Validation overview</a></li>
<li><a href="#sec:validation_control_flow">10.2. Validation control flow</a></li>
<li><a href="#sec:validation_issue_ids">10.3. Issue IDs and Messages</a></li>
<li><a href="#sec:validation_usage_patterns">10.4. Usage Pattern</a></li>
<li><a href="#sec:validation_links">10.5. Links</a></li>
</ul>
</li>
<li><a href="#_references">11. References</a>
<ul class="sectlevel2">
<li><a href="#sec:usecases">11.1. Use cases</a></li>
<li><a href="#sec:calculation_algorithm">11.2. Calculation algorithm</a>
<ul class="sectlevel3">
<li><a href="#sec:Xtext_default_implementation">11.2.1. Xtext default implementation</a></li>
<li><a href="#sec:N4_implementation">11.2.2. N4JS implementation</a></li>
</ul>
</li>
<li><a href="#sec:PerformanceOfDependencyCalculation">11.3. Performance Of Dependency Calculation</a></li>
<li><a href="#sec:kinds_of_references">11.4. Kinds of references</a>
<ul class="sectlevel3">
<li><a href="#sec:Cross_References_to_be_ignored">11.4.1. Cross References to be ignored</a></li>
<li><a href="#sec:Cross_References_to_be_handled">11.4.2. Cross References to be handled</a></li>
</ul>
</li>
<li><a href="#sec:transitive_dependencies">11.5. Transitive dependencies</a></li>
<li><a href="#sec:find-references">11.6. Find references</a>
<ul class="sectlevel3">
<li><a href="#_background">11.6.1. Background</a></li>
<li><a href="#_how-find-references-work">11.6.2. How Find References Work</a>
<ul class="sectlevel4">
<li><a href="#_step-1-convert-cursor-position-to-declared-element">11.6.2.1. Step 1: Convert Cursor Position to Declared Element</a></li>
<li><a href="#_step-2-convert-declared-element-to-target-uris">11.6.2.2. Step 2: Convert Declared Element to Target URIs</a></li>
<li><a href="#_step-3-filter-potential-resources">11.6.2.3. Step 3: Filter Potential Resources</a></li>
<li><a href="#_step-4-search-references-in-resource">11.6.2.4. Step 4: Search References in Resource</a></li>
<li><a href="#_limitations-and-possible-enhancements">11.6.2.5. Limitations and Possible Enhancements</a></li>
</ul>
</li>
</ul>
</li>
</ul>
</li>
<li><a href="#_compilation">12. Compilation</a>
<ul class="sectlevel2">
<li><a href="#chap:compilation">12.1. Introduction</a>
<ul class="sectlevel3">
<li><a href="#sec:general_design_rationals">12.1.1. General design rationals</a>
<ul class="sectlevel4">
<li><a href="#sec:logging_and_error_reporting">12.1.1.1. Logging and error reporting</a></li>
<li><a href="#sec:progress_monitor">12.1.1.2. Progress monitor</a></li>
</ul>
</li>
<li><a href="#sec:Xtext_Integration">12.1.2. Xtext Integration</a>
<ul class="sectlevel4">
<li><a href="#sec:xtext_default_behaviour">12.1.2.1. Xtext default behaviour</a></li>
<li><a href="#sec:n4js_requirements">12.1.2.2. N4JS requirements</a></li>
<li><a href="#sec:compiler_discovery_in_ui">12.1.2.3. Compiler discovery in UI</a></li>
<li><a href="#sec:compiler_discovery_in_headless">12.1.2.4. Compiler discovery in headless</a></li>
<li><a href="#sec:general_generator_implementation">12.1.2.5. General generator implementation</a></li>
<li><a href="#sec:general_generator_activation">12.1.2.6. General generator activation</a></li>
</ul>
</li>
<li><a href="#sec:Overview_of_Input_Models">12.1.3. Overview of the Input Models</a></li>
</ul>
</li>
<li><a href="#sec:Core_Generator">12.2. Generators</a>
<ul class="sectlevel3">
<li><a href="#sec:Compiler_Components">12.2.1. Generator Components</a></li>
<li><a href="#sec:Generator_architecture">12.2.2. Generator architecture</a></li>
<li><a href="#sec:Unified_Compiler_Configuration">12.2.3. Unified Compiler Configuration</a></li>
</ul>
</li>
<li><a href="#sec:Transpilers">12.3. Transpilers</a>
<ul class="sectlevel3">
<li><a href="#sec:Phases">12.3.1. Overview</a></li>
<li><a href="#relation-between-ast-and-im">12.3.2. Relation between AST and IM</a></li>
<li><a href="#implementation-overview">12.3.3. Implementation Overview</a></li>
<li><a href="#sec:Guidelines_for_Implementing_Transformations">12.3.4. Guidelines for Implementing Transformations</a></li>
<li><a href="#symbol-table-in-the-im">12.3.5. Symbol Table in the IM</a></li>
</ul>
</li>
<li><a href="#sec:N4JS_to_EcmaScript_Transpiler">12.4. N4JS-to-EcmaScript Transpiler</a>
<ul class="sectlevel3">
<li><a href="#sec:Overview_of_Transformations">12.4.1. Overview of Transformations</a></li>
<li><a href="#sec:Transpiling_members">12.4.2. Transpiling members</a>
<ul class="sectlevel4">
<li><a href="#sec:Transpiling_members__Delegating_members">12.4.2.1. Techniques for special member handling</a></li>
<li><a href="#sec:Transpiling_members__Partial_shadowing_of_getter_setter_pairs">12.4.2.2. Partial shadowing</a></li>
<li><a href="#sec:Transpiling_members__Consuming_or_inheriting_members_of_an_interface">12.4.2.3. Consuming or inheriting members of an interface</a></li>
<li><a href="#sec:Transpiling_members__Static_polyfill">12.4.2.4. Static polyfill</a></li>
<li><a href="#sec:Transpiling_members__API_implementation_stubs">12.4.2.5. API / implementation stubs</a></li>
</ul>
</li>
<li><a href="#sec:Support_for_incomplete_API_implementation_testing_in_the_N4JS_to_EcmaScript_5_Transpiler">12.4.3. Support for incomplete API implementation testing</a>
<ul class="sectlevel4">
<li><a href="#sec:Modifications_in_Impl_projects">12.4.3.1. Modifications in Impl projects</a></li>
<li><a href="#sec:Implementation_of_stub_generation">12.4.3.2. Implementation of stub-generation</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#sec:n4jsc_Headless_Compiler_Interface">12.5. n4jsc Headless Compiler Interface</a>
<ul class="sectlevel3">
<li><a href="#sec:building_the_headless_compiler">12.5.1. building the headless compiler</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#_execution">13. Execution</a>
<ul class="sectlevel2">
<li><a href="#sec:N4JS_Project_Execution_And_Linking_Model">13.1. N4JS Project Execution And Linking Model</a>
<ul class="sectlevel3">
<li><a href="#subsec:N4JS_Execution_With_NodeJS">13.1.1. N4JS Execution With NodeJS</a></li>
</ul>
</li>
<li><a href="#sec:N4JS_Execution_And_Linking_File">13.2. N4JS Execution And Linking File</a>
<ul class="sectlevel3">
<li><a href="#subsec:NodeJS_Specific_ELF">13.2.1. NodeJS Specific ELF</a></li>
</ul>
</li>
<li><a href="#sec:Runners-execution">13.3. Runners</a>
<ul class="sectlevel3">
<li><a href="#subsec:N4_Runtime_Environments_Convention">13.3.1. N4 Runtime Environments Convention</a></li>
<li><a href="#subsec:Passing_Information_from_IDE_to_Execution_Code_in_Runtime_Environment">13.3.2. Passing Information from IDE to Execution Code in Runtime Environment</a></li>
<li><a href="#subsec:Runners_Design">13.3.3. Runners Design</a></li>
</ul>
</li>
<li><a href="#sec:Legacy_Execution_Engine">13.4. Legacy Execution Engine</a></li>
<li><a href="#sec:Design">13.5. Design</a>
<ul class="sectlevel3">
<li><a href="#sec:Usage_Outside_N4JSIDE">13.5.1. Usage Outside N4JSIDE</a>
<ul class="sectlevel4">
<li><a href="#sec:Use_Node_with_Maven">13.5.1.1. Use Node with Maven</a></li>
</ul>
</li>
<li><a href="#sec:Usage_Inside_N4JSIDE">13.5.2. Usage Inside N4JSIDE</a></li>
</ul>
</li>
<li><a href="#sec:Runtime_Injection">13.6. Runtime Injection</a>
<ul class="sectlevel3">
<li><a href="#sec:Running_String_Code">13.6.1. Running String Code</a></li>
<li><a href="#sec:Running_File_Code">13.6.2. Running File Code</a></li>
<li><a href="#sec:Injection_Code_Example">13.6.3. Injection Code Example</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#_tests">14. Tests</a>
<ul class="sectlevel2">
<li><a href="#sec:Performance_Tests">14.1. Performance Tests</a>
<ul class="sectlevel3">
<li><a href="#sec:Synthetic_Performance_Tests">14.1.1. Synthetic Performance Tests</a>
<ul class="sectlevel4">
<li><a href="#sec:Design_of_Generator">14.1.1.1. Design of Generator</a></li>
<li><a href="#sec:Design_of_Performance_Test_Execution">14.1.1.2. Design of Performance Test Configuration and Execution</a></li>
<li><a href="#sec:JUnit_Configuration">14.1.1.3. JUnit Configuration</a></li>
<li><a href="#sec:JUnitBenchmark_Test_Configuration">14.1.1.4. JUnitBenchmark Test Configuration</a></li>
<li><a href="#sec:JUnitBenchmark_Report_Configuration">14.1.1.5. JUnitBenchmark Report Configuration</a></li>
<li><a href="#sec:JUnitBenchmark_Run_Configuration">14.1.1.6. JUnitBenchmark Run Configuration</a></li>
<li><a href="#sec:JUnitBenchmark_Example">14.1.1.7. JUnitBenchmark Example</a></li>
<li><a href="#sec:Note_on_Jenkins_Job">14.1.1.8. Note on Jenkins Job</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#sec:ECMA_Tests">14.2. ECMA Tests</a>
<ul class="sectlevel3">
<li><a href="#sec:Grammar_Tests">14.2.1. Grammar Tests</a>
<ul class="sectlevel4">
<li><a href="#sec:Negative_Tests">14.2.1.1. Negative Tests</a></li>
<li><a href="#sec:Test_Exclusion">14.2.1.2. Test Exclusion</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#sec:Integration_Tests">14.3. Integration Tests</a></li>
<li><a href="#sec:Test_Helpers">14.4. Test Helpers</a>
<ul class="sectlevel3">
<li><a href="#sec:Parameterized_N4JS_Tests">14.4.1. Parameterized N4JS tests</a>
<ul class="sectlevel4">
<li><a href="#sec:ParameterizedXtextRunner">14.4.1.1. ParameterizedXtextRunner</a></li>
<li><a href="#sec:TestCodeProvider">14.4.1.2. TestCodeProvider</a></li>
<li><a href="#sec:Example_Of_Parameterized_Parser_Test">14.4.1.3. Example of parameterized parser test</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#sec:Issue_Suppression">14.5. Issue Suppression</a></li>
<li><a href="#sec:Xpect_Tests">14.6. Xpect Tests</a>
<ul class="sectlevel3">
<li><a href="#sec:Xpect_Test_Setup">14.6.1. Xpect Test Setup</a></li>
<li><a href="#sec:Xpect_Issue_Suppression">14.6.2. Xpect Issue Suppression</a></li>
<li><a href="#sec:Xpect_Provided_Test_Methods">14.6.3. Xpect Provided Test Methods</a>
<ul class="sectlevel4">
<li><a href="#errors">14.6.3.1. errors</a></li>
<li><a href="#warnings">14.6.3.2. warnings</a></li>
</ul>
</li>
<li><a href="#sec:N4JS_Specific_Xpect_Test_Methods">14.6.4. N4JS Specific Xpect Test Methods</a>
<ul class="sectlevel4">
<li><a href="#sec:XPECT_noerrors">14.6.4.1. noerrors and nowarnings</a></li>
<li><a href="#sec:XPECT_scope">14.6.4.2. scope</a></li>
<li><a href="#sec:XPECT_scopeWithPosition">14.6.4.3. scopeWithPosition</a></li>
<li><a href="#sec:XPECT_scopeWithResource">14.6.4.4. scopeWithResource</a></li>
<li><a href="#sec:XPECT_binding">14.6.4.5. binding</a></li>
<li><a href="#sec:XPECT_linkedPathname">14.6.4.6. linkedPathname</a></li>
<li><a href="#sec:XPECT_type_of">14.6.4.7. type of</a></li>
<li><a href="#sec:XPECT_expectedType">14.6.4.8. expectedType</a></li>
<li><a href="#sec:XPECT_elementKeyword">14.6.4.9. elementKeyword</a></li>
<li><a href="#sec:XPECT_accessModifier">14.6.4.10. accessModifier</a></li>
<li><a href="#sec:XPECT_compileResult">14.6.4.11. compileResult</a></li>
<li><a href="#sec:XPECT_output">14.6.4.12. output</a></li>
<li><a href="#sec:XPECT_outputRegEx">14.6.4.13. outputRegEx</a></li>
<li><a href="#sec:XPECT_calculatedAccessModifier">14.6.4.14. calculatedAccessModifier</a></li>
<li><a href="#sec:XPECT_spec">14.6.4.15. spec</a></li>
<li><a href="#sec:XPECT_deadCode">14.6.4.16. deadCode</a></li>
<li><a href="#sec:XPECT_returnOrThrows">14.6.4.17. returnOrThrows</a></li>
<li><a href="#sec:XPECT_lint">14.6.4.18. lint</a></li>
<li><a href="#sec:XPECT_lintFails">14.6.4.19. lintFails</a></li>
</ul>
</li>
<li><a href="#sec:FIXME_Xpect_modifier">14.6.5. FIXME Xpect modifier</a></li>
<li><a href="#sec:Expectmatrix_Xpect_Test_Methods">14.6.6. Expectmatrix Xpect tests</a></li>
<li><a href="#xpect-lint-tests">14.6.7. Xpect Lint Tests</a></li>
</ul>
</li>
<li><a href="#xpect-proposal-tests">14.7. Xpect Proposal Tests</a>
<ul class="sectlevel3">
<li><a href="#sec:Validation_vs__Non_Validation">14.7.1. Validation vs. Non-Validation</a></li>
<li><a href="#sec:General_Proposal_Test_Features">14.7.2. General Proposal Test Features</a>
<ul class="sectlevel4">
<li><a href="#sec:Test_Variables">14.7.2.1. Test Variables</a></li>
<li><a href="#sec:Location_and_Selection">14.7.2.2. at – Location and Selection</a></li>
<li><a href="#sec:Multi_Line_Expectations_in_Proposal_Tests">14.7.2.3. Multi Line Expectations in Proposal Tests</a></li>
<li><a href="#sec:Timeout">14.7.2.4. Timeout and Performance</a></li>
</ul>
</li>
<li><a href="#proposals-verify-existence-of-proposals">14.7.3. proposals – Verify Existence of Proposals</a></li>
<li><a href="#sec:Verify_displayed_string">14.7.4. display – Verify displayed string</a></li>
<li><a href="#sec:Apply_Proposal">14.7.5. apply – Apply Proposal</a>
<ul class="sectlevel4">
<li><a href="#resource-application-in-other-files">14.7.5.1. resource – application in other files</a></li>
</ul>
</li>
<li><a href="#sec:Content_Assist_Cycling">14.7.6. kind – Content Assist Cycling</a></li>
<li><a href="#fileValidVerify-validation-status">14.7.7. fileValid – Verify validation status</a></li>
</ul>
</li>
<li><a href="#sec:Apply_Proposal_And_Execute_Tests">14.8. Apply Proposal And Execute Tests</a></li>
<li><a href="#sec:Organize_Imports_Test">14.9. Organize Imports Test</a>
<ul class="sectlevel3">
<li><a href="#organizeimports">14.9.1. organizeImports</a></li>
</ul>
</li>
<li><a href="#sec:Access_Control_Test">14.10. Access Control Test</a>
<ul class="sectlevel3">
<li><a href="#test-scenarios">14.10.1. Test Scenarios</a></li>
<li><a href="#n4js-code-generator">14.10.2. N4JS Code Generator</a></li>
<li><a href="#xtext-issue-matcher">14.10.3. Xtext Issue Matcher</a></li>
</ul>
</li>
<li><a href="#sec:Smoke_Tests">14.11. Smoke Tests</a>
<ul class="sectlevel3">
<li><a href="#how-to-handle-smoke-test-errors">14.11.1. How to handle smoke test errors?</a></li>
<li><a href="#smoketester-and-exceptionanalyzer">14.11.2. SmokeTester and ExceptionAnalyzer</a></li>
</ul>
</li>
<li><a href="#sec:UI_Tests_with_SWTBot">14.12. UI Tests with SWTBot</a>
<ul class="sectlevel3">
<li><a href="#writing-swtbot-tests">14.12.1. Writing SWTBot Tests</a></li>
<li><a href="#running-swtbot-tests">14.12.2. Running SWTBot Tests</a></li>
</ul>
</li>
<li><a href="#sec:Debugging_UI_Tests">14.13. Debugging UI Tests</a>
<ul class="sectlevel3">
<li><a href="#sec:Connecting_to_the_X_server_on_the_build_node">14.13.1. Connecting to the X-server on the build-node</a></li>
<li><a href="#sec:Tools_to_investigate_the_java_stack">14.13.2. Tools to investigate the java-stack</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#_ui-concepts">15. UI Concepts</a>
<ul class="sectlevel2">
<li><a href="#sec:User_Interface_Concepts">15.1. User Interface Concepts</a>
<ul class="sectlevel3">
<li><a href="#sec:Eclipse_UI_Concepts">15.1.1. Eclipse UI Concepts</a>
<ul class="sectlevel4">
<li><a href="#sec:Label_Provider">15.1.1.1. Label Provider</a></li>
<li><a href="#sec:Markers">15.1.1.2. Markers</a></li>
<li><a href="#sec:Commands__Toolbar_and_Menus">15.1.1.3. Commands, Toolbar and Menus</a></li>
<li><a href="#sec:Content_Assist">15.1.1.4. Content Assist</a></li>
<li><a href="#sec:Quick_Fixes">15.1.1.5. Quick Fixes</a></li>
<li><a href="#sec:Quick_Assist">15.1.1.6. Quick Assist</a></li>
<li><a href="#sec:Clean_Up_Actions">15.1.1.7. Clean Up Actions</a></li>
<li><a href="#sec:Save_Actions">15.1.1.8. Save Actions</a></li>
<li><a href="#sec:Auto_Edit">15.1.1.9. Auto Edit</a></li>
<li><a href="#sec:Template_Proposals">15.1.1.10. Template Proposals</a></li>
<li><a href="#sec:Outline_View___Quick_Outline">15.1.1.11. Outline View / Quick Outline</a></li>
<li><a href="#sec:Navigator__Package_Explorer__Project_Explorer">15.1.1.12. Navigator, Package Explorer, Project Explorer</a></li>
<li><a href="#sec:Hyperlinking_and_Navigation">15.1.1.13. Hyperlinking and Navigation</a></li>
<li><a href="#sec:Syntax_and_Semantic_Coloring">15.1.1.14. Syntax and Semantic Coloring</a></li>
<li><a href="#sec:Code_Formatter">15.1.1.15. Code Formatter</a></li>
<li><a href="#sec:Wizards">15.1.1.16. Wizards</a></li>
<li><a href="#sec:Cheat_Sheets">15.1.1.17. Cheat Sheets</a></li>
<li><a href="#sec:Context_sensitive_Help">15.1.1.18. Context-sensitive Help</a></li>
<li><a href="#sec:Hovers">15.1.1.19. Hovers</a></li>
<li><a href="#sec:Folding">15.1.1.20. Folding</a></li>
<li><a href="#sec:Customizable_validation___severity">15.1.1.21. Customizable validation / severity</a></li>
<li><a href="#sec:Proposals">15.1.1.22. Proposals</a></li>
</ul>
</li>
<li><a href="#sec:Non_Eclipse_UI_Concepts">15.1.2. Non-Eclipse UI Concepts</a>
<ul class="sectlevel4">
<li><a href="#sec:Overlays">15.1.2.1. Overlays</a></li>
<li><a href="#sec:Goto__Inferred__Type">15.1.2.2. Goto (Inferred) Type</a></li>
<li><a href="#sec:Postfix_Completion">15.1.2.3. Postfix Completion</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#_user-interface-resources">15.2. User Interface Resources</a>
<ul class="sectlevel3">
<li><a href="#_icons">15.2.1. Icons</a>
<ul class="sectlevel4">
<li><a href="#_eclipse-platform-icons">15.2.1.1. Eclipse Platform Icons</a></li>
<li><a href="#_n4js-specific-icons">15.2.1.2. N4JS Specific Icons</a></li>
<li><a href="#_high-resolution-icons">15.2.1.3. High Resolution Icons</a></li>
</ul>
</li>
</ul>
</li>
</ul>
</li>
<li><a href="#_formatting">16. Formatting</a>
<ul class="sectlevel2">
<li><a href="#sec:FmtObjective">16.1. Objective</a>
<ul class="sectlevel3">
<li><a href="#sec:FmtFormatting_Comments">16.1.1. Formatting Comments</a></li>
</ul>
</li>
<li><a href="#sec:FmtArchitecture">16.2. Architecture</a>
<ul class="sectlevel3">
<li><a href="#sec:Implementation_example">16.2.1. Implementation example</a></li>
</ul>
</li>
<li><a href="#sec:FmtFormatter_Implementation_Guidelines">16.3. Formatter Implementation Guidelines</a></li>
<li><a href="#sec:FmtConfiguration">16.4. Configuration</a></li>
<li><a href="#sec:FmtUI_Integration">16.5. UI Integration</a></li>
<li><a href="#sec:FmtUnit_Testing_with_Xpect">16.6. Unit Testing with Xpect</a></li>
</ul>
</li>
<li><a href="#_external-libraries">17. External Libraries</a>
<ul class="sectlevel2">
<li><a href="#sec:Major_Components">17.1. Major Components</a>
<ul class="sectlevel3">
<li><a href="#subsec:External_Resources">17.1.1. External Resources</a></li>
<li><a href="#subsec:External_Library_Workspace">17.1.2. External Library Workspace</a></li>
<li><a href="#subsec:External_Library_Preference_Store">17.1.3. External Library Preference Store</a></li>
<li><a href="#subsec:npm_Manager">17.1.4. Library Manager</a></li>
<li><a href="#subsec:External_Library_Builder_Helper">17.1.5. External Library Builder</a></li>
<li><a href="#subsec:External_Library_Xtext_Index_Persister">17.1.6. External Library Xtext Index Persister</a></li>
<li><a href="#subsec:External_Library_Preference_Page">17.1.7. External Library Preference Page</a></li>
</ul>
</li>
<li><a href="#sec:Headless_External_Library_Support">17.2. Headless External Library Support</a>
<ul class="sectlevel3">
<li><a href="#_custom-npm-settings">17.2.1. Custom npm settings</a></li>
</ul>
</li>
<li><a href="#sec:lmFutureWork">17.3. Future Work</a>
<ul class="sectlevel3">
<li><a href="#subsec:lmMultipleDependencyScope">17.3.1. Multiple Dependency Scope</a></li>
<li><a href="#subsec:lmRunTestsFromLibrary">17.3.2. Run Tests from TestLibrary</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#sec:JSON_Support">18. JSON Support</a>
<ul class="sectlevel2">
<li><a href="#sec:JSON_Parser">18.1. JSON Parser</a>
<ul class="sectlevel3">
<li><a href="#sec:JSON_Parser_Unicode_Escaping">18.1.1. Escaping Unicode Control Characters in String Literals</a></li>
<li><a href="#sec:JSON_Parser_Empty_Text">18.1.2. Empty Text</a></li>
<li><a href="#sec:JSON_Parser_Nested_Structures">18.1.3. Nested Structures</a></li>
<li><a href="#sec:JSON_Parser_Whitespace">18.1.4. Whitespace</a></li>
<li><a href="#sec:JSON_Parser_Comments">18.1.5. Comments</a></li>
</ul>
</li>
<li><a href="#sec:JSON_Language_Extensions">18.2. JSON Language Extensions</a>
<ul class="sectlevel3">
<li><a href="#sec:JSON_Validator_Extensions">18.2.1. JSON Validator Extensions</a>
<ul class="sectlevel4">
<li><a href="#sec:File_Specitic_Validator_Extensions">18.2.1.1. File-Specific Validator Extensions</a></li>
<li><a href="#sec:JSON_Declarative_JSON_Validator_Extensions">18.2.1.2. Declarative JSON Validator Extensions</a></li>
</ul>
</li>
<li><a href="#_json-resource-description-strategy">18.2.2. JSON Resource Description Strategy</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#_jsdoc">19. JSDoc</a>
<ul class="sectlevel2">
<li><a href="#sec:Design_Rationale">19.1. Design Rationale</a>
<ul class="sectlevel3">
<li><a href="#_general-design">19.1.1. General Design</a></li>
<li><a href="#sec:Type_Expressions">19.1.2. Type Expressions</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#_docexporter">20. DocExporter</a>
<ul class="sectlevel2">
<li><a href="#sec:Specification_Exporter">20.1. Specification Exporter</a></li>
</ul>
</li>
<li><a href="#_rename-refactoring">21. Rename Refactoring</a>
<ul class="sectlevel2">
<li><a href="#_rename-refactoring-ui-interaction">21.1. Rename Refactoring UI interaction</a></li>
<li><a href="#_renameelementprocessor-interaction">21.2. RenameElementProcessor interaction</a></li>
</ul>
</li>
<li><a href="#chap:flowgraphs">22. Flow Graphs</a>
<ul class="sectlevel2">
<li><a href="#sec:flowgraphs_overview">22.1. Flow graphs overview</a>
<ul class="sectlevel3">
<li><a href="#_internal-graph">22.1.1. Internal graph</a></li>
<li><a href="#_optimizations">22.1.2. Optimizations</a></li>
<li><a href="#_api-for-client-analyses">22.1.3. API for client analyses</a>
<ul class="sectlevel4">
<li><a href="#_mapping-from-internal-to-ast-elements">22.1.3.1. Mapping from internal to AST elements</a></li>
<li><a href="#_graph-visitor">22.1.3.2. Graph visitor</a></li>
<li><a href="#_graph-explorer">22.1.3.3. Graph explorer</a></li>
<li><a href="#_branch-walker">22.1.3.4. Branch walker</a></li>
<li><a href="#_example-1-compute-string-for-each-path">22.1.3.5. Example 1: Compute string for each path</a></li>
<li><a href="#_path-quantor">22.1.3.6. Path quantor</a></li>
</ul>
</li>
<li><a href="#_control-flow-analyses">22.1.4. Control flow analyses</a>
<ul class="sectlevel4">
<li><a href="#_dead-code-analysis">22.1.4.1. Dead code analysis</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#sec:dataflow">22.2. Dataflow</a>
<ul class="sectlevel3">
<li><a href="#_dataflow-graph">22.2.1. Dataflow graph</a></li>
<li><a href="#_dataflow-analyses">22.2.2. Dataflow analyses</a>
<ul class="sectlevel4">
<li><a href="#_def-def-def-nothing-analysis">22.2.2.1. Def&#8594;Def / Def&#8594;Nothing analysis</a></li>
<li><a href="#_def-use-decl-analysis">22.2.2.2. Def|Use&#8592;Decl analysis</a></li>
</ul>
</li>
</ul>
</li>
</ul>
</li>
<li><a href="#sec:publish-npms-to-public">23. Publish npms</a>
<ul class="sectlevel2">
<li><a href="#sec:publish-npms-n4js-maven">23.1. Publish n4js-libs to during maven build</a></li>
</ul>
</li>
<li><a href="#sec:Hints">Appendix A: Hints</a>
<ul class="sectlevel2">
<li><a href="#sec:XtextInjection">A.1. Xtext Injection</a>
<ul class="sectlevel3">
<li><a href="#sec:DI_MultipleInjectors_Singletons">A.1.1. Multiple Injectors and Singletons</a>
<ul class="sectlevel4">
<li><a href="#sec:DI_avoid_duplicate_singletons">A.1.1.1. Avoiding duplicate singletons</a>
<ul class="sectlevel5">
<li><a href="#sec:DI_binding_in_shared">A.1.1.1.1. Defining binding in the shared injector</a></li>
<li><a href="#sec:DI_binding_in_custom">A.1.1.1.2. Defining binding in the custom injector</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#sec:DI_Hints">A.1.2. Dependency Injection Hints</a>
<ul class="sectlevel4">
<li><a href="#sec:DI_custom_bundle">A.1.2.1. Use DI in custom bundle, use DI with extensions</a>
<ul class="sectlevel5">
<li><a href="#sec:DI_custom_bundle_problem">A.1.2.1.1. Problem</a></li>
<li><a href="#sec:DI_custom_bundle_solution">A.1.2.1.2. Solution</a></li>
</ul>
</li>
<li><a href="#sec:Access_Other_DSL_Injector">A.1.2.2. How do I get the Guice Injector of my language?</a>
<ul class="sectlevel5">
<li><a href="#sec:DSL_Injector_UI_context">A.1.2.2.1. UI context</a></li>
<li><a href="#sec:DSL_Injector_Non_UI_context">A.1.2.2.2. Non UI context but with injection context</a></li>
<li><a href="#sec:DSL_Injector_Non_UI_non_injection_context">A.1.2.2.3. Non UI context without injection context</a></li>
</ul>
</li>
<li><a href="#sec:Cancel_Indicator">A.1.2.3. How do I get cancel indicators in different contexts?</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#sec:Eclipse">A.2. Eclipse</a>
<ul class="sectlevel3">
<li><a href="#sec:Show_Xtext_Index">A.2.1. Show the current Xtext index</a></li>
<li><a href="#sec:Plugin_spy">A.2.2. Plug-in spy</a></li>
</ul>
</li>
<li><a href="#sec:Maven-hints">A.3. Maven</a>
<ul class="sectlevel3">
<li><a href="#how-to-check-for-maven-mojo-updates">A.3.1. How to check for Maven MOJO updates</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#_module-loading">Appendix B: Module Loading</a>
<ul class="sectlevel2">
<li><a href="#sec:Dependency_Management">B.1. Dependency Management</a></li>
<li><a href="#ecmascript-modules">B.2. ECMAScript Modules</a>
<ul class="sectlevel3">
<li><a href="#sec:ES5_Modules_Systems">B.2.1. ES5 Modules Systems</a></li>
<li><a href="#sec:ES6_Modules">B.2.2. ES6 Modules</a></li>
</ul>
</li>
<li><a href="#sec:ECMAScript_Module_Loaders">B.3. ECMAScript Module Loaders</a>
<ul class="sectlevel3">
<li><a href="#sec:ES6_Module_Loaders">B.3.1. ES6 Module Loaders</a></li>
<li><a href="#sec:Polyfills_for_ES6_Module_Loaders">B.3.2. Polyfills for ES6 Module Loaders</a>
<ul class="sectlevel4">
<li><a href="#sec:es6_module_loader">B.3.2.1. es6-module-loader</a></li>
<li><a href="#sec:SystemJS">B.3.2.2. SystemJS</a></li>
<li><a href="#sec:Demo">B.3.2.3. Demo</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#case-study-typescript">B.4. Case Study : TypeScript</a>
<ul class="sectlevel3">
<li><a href="#sec:ES6_Modules_Support">B.4.1. ES6 Modules Support</a></li>
<li><a href="#sec:TypeScript_and_Module_Loading">B.4.2. TypeScript and Module Loading</a></li>
</ul>
</li>
<li><a href="#sec:Cyclic_Dependencies">B.5. Cyclic Dependencies</a>
<ul class="sectlevel3">
<li><a href="#sec:Setup">B.5.1. Setup</a></li>
<li><a href="#sec:Transpile_and_Execute">B.5.2. Transpile and Execute</a>
<ul class="sectlevel4">
<li><a href="#sec:Module_Format___AMD">B.5.2.1. Module Format = AMD</a></li>
<li><a href="#sec:Module_Format___CommonJS">B.5.2.2. Module Format = CommonJS</a></li>
<li><a href="#sec:Module_Format___SystemJS">B.5.2.3. Module Format = SystemJS</a></li>
</ul>
</li>
<li><a href="#sec:Conclusion">B.5.3. Conclusion</a></li>
</ul>
</li>
<li><a href="#system.register-as-transpilation-target">B.6. System.register as transpilation target</a>
<ul class="sectlevel3">
<li><a href="#sec:Introduction">B.6.1. Introduction</a>
<ul class="sectlevel4">
<li><a href="#sec:External_Transpilers">B.6.1.1. External Transpilers</a></li>
<li><a href="#sec:Example_of_a_System_register_module">B.6.1.2. Example of a System.register module</a></li>
</ul>
</li>
<li><a href="#sec:Structure_of_a_System_register_module">B.6.2. Structure of a System.register module</a></li>
<li><a href="#_transpilation-hints">B.6.3. Transpilation Hints</a>
<ul class="sectlevel4">
<li><a href="#sec:Handling_Imports">B.6.3.1. Handling Imports</a></li>
<li><a href="#sec:__exportFn__">B.6.3.2. &lt;&lt;exportFn&gt;&gt;</a></li>
<li><a href="#sec:Handling_Exports">B.6.3.3. Handling Exports</a></li>
</ul>
</li>
<li><a href="#sec:Examples_w__Circular_Dependencies">B.6.4. Examples w/ Circular Dependencies</a></li>
<li><a href="#sec:N4JS_Examples_w__Circular_Dependencies">B.6.5. N4JS Examples w/ Circular Dependencies</a>
<ul class="sectlevel4">
<li><a href="#sec:Unresolved_Cyclic_Dependencies">B.6.5.1. Unresolved Cyclic Dependencies</a></li>
<li><a href="#sec:Variables___Functions">B.6.5.2. Examples with Variables &amp; Functions</a></li>
<li><a href="#sec:Classes">B.6.5.3. Examples with Classes</a></li>
<li><a href="#sec:Examples_with_SubClassing">B.6.5.4. Examples with SubClassing</a></li>
<li><a href="#sec:Miscellaneous">B.6.5.5. Miscellaneous</a></li>
</ul>
</li>
<li><a href="#_resources">B.6.6. Resources</a></li>
</ul>
</li>
<li><a href="#sec:CommonJS_as_transpilation_target">B.7. CommonJS as transpilation target</a>
<ul class="sectlevel3">
<li><a href="#_introduction-2">B.7.1. Introduction</a></li>
<li><a href="#sec:Transpilation_Hints">B.7.2. Transpilation Hints</a>
<ul class="sectlevel4">
<li><a href="#sec:Import_Statements">B.7.2.1. Import Statements</a></li>
<li><a href="#sec:Export_Statements">B.7.2.2. Export Statements</a></li>
<li><a href="#sec:Tracking_Live_Bindings">B.7.2.3. Tracking Live Bindings</a></li>
<li><a href="#sec:A_complete_example">B.7.2.4. A complete example</a></li>
</ul>
</li>
<li><a href="#_resources-2">B.7.3. Resources</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#sec:License">Appendix C: License</a></li>
<li><a href="#sec:Acronyms">Appendix D: Acronyms</a></li>
<li><a href="#_bibliography-and-footnotes">Appendix E: Bibliography and Footnotes</a></li>
</ul>
</div>
</div>
<div id="content">
<div id="preamble">
<div class="sectionbody">
<div class="paragraph center">
<p><strong>Last Updated: 2019-08-08</strong></p>
</div>
<div class="paragraph center">
<p><strong>Authors:</strong><br>
Jens von Pilgrim, Jakub Siberski, Mark-Oliver Reiser, Torsten Krämer, Ákos Kitta, Sebastian Zarnekow, Lorenzo Bettini, Jörg Reichert, Kristian Duske, Marcus Mews, Minh Quang Tran, Luca Beurer-Kellner</p>
</div>
<div style="page-break-after: always;"></div>
<div class="paragraph">
<p>This document contains the N4JS Design and Implementation documentation.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_introduction"><a class="anchor" href="#_introduction"></a><a class="link" href="#_introduction">1. Introduction</a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>This document describes design aspects of the N4JS compiler and IDE. It relies on the following N4JS related specifications:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>N4JS Language Specification [<a href="#N4JSSpec">N4JSSpec</a>]</p>
</li>
</ul>
</div>
<div class="sect2">
<h3 id="notation"><a class="anchor" href="#notation"></a><a class="link" href="#notation">1.1. Notation</a></h3>
<div class="paragraph">
<p>We reuse the notation specified in [<a href="#N4JSSpec">N4JSSpec</a>].</p>
</div>
</div>
<div class="sect2">
<h3 id="sec:IDE_Overview"><a class="anchor" href="#sec:IDE_Overview"></a><a class="link" href="#sec:IDE_Overview">1.2. IDE Components</a></h3>
<div class="paragraph">
<p>The N4JS and N4JSIDE components are organized via features. The following features with included plugins are defined
(the common prefix "org.eclipse.n4js" is omitted at the plugin name):</p>
</div>
<table class="tableblock frame-all grid-all spread">
<colgroup>
<col style="width: 5.8823%;">
<col style="width: 11.7647%;">
<col style="width: 82.353%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Feature</th>
<th class="tableblock halign-left valign-top">Plugin</th>
<th class="tableblock halign-left valign-top">Description</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock"><strong>org.eclipse.n4js.lang.sdk</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">N4JS core language with parser, validation etc.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">org.eclipse.n4js</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Xtext grammar with generator and custom code for N4JS, scoping (and binding) implementation, basic validation (and Xsemantics type system).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">doc</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">(in doc folder) General documentation (including web page) written in AsciiDoc</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">external.libraries</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Support for N4JS libraries shipped with the IDE, i.e. core N4JS library and mangelhaft.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">ui</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">UI components for N4JS, e.g., proposal provider, labels, outline, quickfixes.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">jsdoc</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Parser and model for JSDoc</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">external.libraries.update</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Not included in feature</strong>. Updates the external library plugin</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock"><strong>org.eclipse.n4js.ts.sdk</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Type System</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">ts</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Xtext grammar with generator and custom code for type expressions and standalone type definitions.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">ts.model</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Xcore based types model with helper classes etc.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">ts.ui</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Xtext generated UI for type system, not really used as this TS files are not editable by users.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock"><strong>org.eclipse.n4js.unicode.sdk</strong></p></td>
<td class="tableblock halign-left valign-top"></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">common.unicode</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Xtext grammar with generator and custom code used by all other grammars for proper unicode support.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock"><strong>org.eclipse.n4js.regex.sdk</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Regular expression grammar and UI, used by N4JS grammar and UI</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">regex</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Xtext grammar with generator and custom code used by N4JS grammars for regular expressions.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">regex.ui</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">UI components for regular expressions, e.g., proposal provider, labels, outline, quickfixes.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock"><strong>org.eclipse.n4js.sdk</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This feature defines the N4JSIDE. It contains core UI plugins and all includes (almost all) other features!</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">environments</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Utility plugin, registers n4scheme for EMF proxy resolution.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">model</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Xcore based N4JS model with helper classes etc.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">product</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">N4JSIDE main application.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">releng.utils</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">(in releng folder) Contains utility classes only used for building the system, e.g., tools for generating antlr based parser with extended features.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">utils</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">general utilities</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">utils.ui</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">general UI utilities</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock"><strong>org.eclipse.n4js.compiler.sdk</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Compilers and Transpilers</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">generator.common</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Not included in feature, logically associated.</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">generator.headless</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">N4JS headless generator (i.e. command line compiler).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">transpiler</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Generic transpiler infrastructure</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">transpiler.es</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Transpiler to compile to EcmaScript</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock"><strong>org.eclipse.n4js.json.sdk</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">N4JS JSON</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">json</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Xtext grammar with generator and custom code for a extensible JSON language support. Used in N4JS for the project description in terms of a <code>package.json</code> file.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">json.ui</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">UI components for extensible JSON language support, e.g., proposal provider, labels, outline.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">json.model</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Not included in feature, logically associated.</strong> Xcore based model for the JSON language.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock"><strong>org.eclipse.n4js.semver.sdk</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Semantic version string support.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">semver</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Parser and tools for semantic version strings.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">semver.ui</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">UI tools for semantic version strings.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">semver.model</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Not included in feature, logically associated.</strong> Xcore model of semantic version strings.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock"><strong>org.eclipse.n4js.runner.sdk</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Runners for executing N4JS or JavaScript code</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">runner</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Generic interfaces and helper for runners, i.e. JavaScript engines executing N4JS or JavaScript code.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">runner.chrome</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Runner for executing N4JS or JavaScript with Chrome.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">runner.chrome.ui</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">UI classes for launching the Chrome runner via the org.eclipse.debug.ui</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">runner.nodejs</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Runner for executing N4JS or JavaScript with node.js.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">runner.nodejs.ui</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">UI classes for launching the node.js runner via the org.eclipse.debug.ui</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">runner.ui</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Generic interfaces for configuring N4JS runner via the debug ui.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock"><strong>org.eclipse.n4js.tester.sdk</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Runners and UI for tests (via mangelhaft).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">tester</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Generic interfaces and helper for testers, i.e. JavaScript engines executing N4JS tests (using mangelhaft).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">tester.nodejs</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Tester based on the nodejs runner for executing mangelhaft tests with node.js</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">tester.nodejs.ui</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">UI for showing test results.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">tester.ui</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Configuration of tests via the debug UI.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock"><strong>org.eclipse.n4js.jsdoc2spec.sdk</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">JSDoc 2 Specification</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">jsdoc2spec</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Exporter to generate API documentation with specification tests awareness</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">jsdoc2spec.ui</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">UI for API doc exporter</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock"><strong>org.eclipse.n4js.xpect.sdk</strong></p></td>
<td class="tableblock halign-left valign-top"></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">xpect</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Xpect test methods.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">xpect.ui</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">UI for running Xpext tests methods from the N4JSIDE (for creating bug reports).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock"><strong>org.eclipse.n4js.smith.sdk</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Feature for internal N4JS IDE plugins only intended for development (for example, the AST Graph view).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">smith</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Non-UI classes for tools for smiths, that is, tools for developers of the N4JS IDE such as AST views etc.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">smith.ui</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">UI classes for tools for smiths, that is, tools for developers of the N4JS IDE such as AST views etc.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock"><strong>org.eclipse.n4js.tests.helper.sdk</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Test helpers.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock"><strong>org.eclipse.n4js.dependencies.sdk</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Collection of all external non-ui dependencies, used for local mirroring of update sites.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock"><strong>org.eclipse.n4js.dependencies.ui.sdk</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Collection of all external ui dependencies, used for local mirroring of update sites.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="3"><p class="tableblock"><strong>uncategorized plugins</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">flowgraphs</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Control and data flow graph model and computer.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock"><strong>Fragments</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">not associated to features, only listed here for completeness</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">utils.logging</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Fragment only, configuration for loggers, in particular for the product and for the tests</p></td>
</tr>
</tbody>
</table>
<div class="sect3">
<h4 id="sec:Naming_Conventions"><a class="anchor" href="#sec:Naming_Conventions"></a><a class="link" href="#sec:Naming_Conventions">1.2.1. Naming Conventions</a></h4>
<div class="paragraph">
<p>In the above sections, tests were omitted. We use the following naming conventions (by example) for test and tests helper:</p>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
project
</td>
<td class="hdlist2">
<p>-</p>
</td>
</tr>
<tr>
<td class="hdlist1">
project.tests
</td>
<td class="hdlist2">
<p>tests for project, is a fragment</p>
</td>
</tr>
<tr>
<td class="hdlist1">
project.tests.helper
</td>
<td class="hdlist2">
<p>helper classes used ONLY by tests</p>
</td>
</tr>
<tr>
<td class="hdlist1">
project.tests.performance
</td>
<td class="hdlist2">
<p>performance tests</p>
</td>
</tr>
<tr>
<td class="hdlist1">
project.tests.integration
</td>
<td class="hdlist2">
<p>integration tests</p>
</td>
</tr>
<tr>
<td class="hdlist1">
project.ui
</td>
<td class="hdlist2">
<p>-</p>
</td>
</tr>
<tr>
<td class="hdlist1">
project.ui.tests
</td>
<td class="hdlist2">
<p>tests for ui project, fragment of project.ui</p>
</td>
</tr>
<tr>
<td class="hdlist1">
project.ui.tests.helper
</td>
<td class="hdlist2">
<p>helper classes used ONLY by tests</p>
</td>
</tr>
<tr>
<td class="hdlist1">
project.ui.tests.performance
</td>
<td class="hdlist2">
<p>-</p>
</td>
</tr>
<tr>
<td class="hdlist1">
tests.helper
</td>
<td class="hdlist2">
<p>general test helper</p>
</td>
</tr>
<tr>
<td class="hdlist1">
ui.tests.helper
</td>
<td class="hdlist2">
<p>general ui test helper</p>
</td>
</tr>
<tr>
<td class="hdlist1">
project.xpect.tests
</td>
<td class="hdlist2">
<p>xpect tests for the project, despite dependnecies to UI the can be executed as plain JUnit tests</p>
</td>
</tr>
<tr>
<td class="hdlist1">
project.xpect.ui.tests
</td>
<td class="hdlist2">
<p>xpect tests for the project, need to be executed as eclipse plugin tests</p>
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>Due to Maven, tests are in subfolder tests (incl. helpers), implementation bundles in plugins, and release engineering related bundles in releng.</p>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_eclipse-setup"><a class="anchor" href="#_eclipse-setup"></a><a class="link" href="#_eclipse-setup">2. Eclipse Setup</a></h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="_system-requirements"><a class="anchor" href="#_system-requirements"></a><a class="link" href="#_system-requirements">2.1. System Requirements</a></h3>
<div class="paragraph">
<p>In all cases, <a href="https://adoptopenjdk.net/">Java 11</a> is required to be installed on your system. <a href="https://nodejs.org/en/download/">Node.js</a> version 10+ is also required, and for some tests you need <a href="https://yarnpkg.com">Yarn</a> to be globally installed.</p>
</div>
</div>
<div class="sect2">
<h3 id="_contribute"><a class="anchor" href="#_contribute"></a><a class="link" href="#_contribute">2.2. Contribute</a></h3>
<div class="paragraph">
<p>Eclipse developers who want to develop N4JS itself should use the <a href="https://www.eclipse.org/downloads/">Oomph Eclipse installer</a>. The N4JS project is listed under "Eclipse Projects/N4JS"
This setup installs the correct Eclipse version, creates a new workspace and clones all projects into it (for details see below).</p>
</div>
<div class="sect3">
<h4 id="_eclipse-installer"><a class="anchor" href="#_eclipse-installer"></a><a class="link" href="#_eclipse-installer">2.2.1. Eclipse Installer</a></h4>
<div class="paragraph">
<p>The recommended way to install the Eclipse IDE and set up the workspace is to use the Eclipse Installer.
This installer is to be downloaded from <a href="https://wiki.eclipse.org/Eclipse_Installer" class="bare">https://wiki.eclipse.org/Eclipse_Installer</a></p>
</div>
<div class="paragraph">
<p>Run the installer and apply the following steps:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>change to "Advance Mode" via the menu (upper-right corner) (no need to move the installer)</p>
</li>
<li>
<p>select a product, e.g. "Eclipse IDE for Eclipse Committers" with product version "2019-06". Hint: Do not select "latest" because this will cause automatic updates which may lead to weird errors later on.</p>
</li>
<li>
<p>double-click the entry <strong>Eclipse Projects/N4JS</strong> so that it is shown in the catalog view below</p>
</li>
<li>
<p>on the next page, configure paths accordingly. You only have to configure the installation and workspace folder. You may want to use git with https instead of ssh.</p>
</li>
<li>
<p>start installation</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>The installer will then guide you through the rest of the installation. All plug-ins are downloaded and configured automatically, so is the workspace including downloading the git repository and setting up the workspace.</p>
</div>
<div class="paragraph">
<p>The workspace is configured automatically. This includes fetching the necessary git repository. If you have selected git with SSH you may run into problems. In this case you can re-run the scripts and select HTTPS instead, this should work in any case.</p>
</div>
<div class="paragraph">
<p>Eventually the installer scripts are done, that means the git repository has been cloned and the workspace has been configured (including the project set setup).
Now the automatic build kicks in as you can see in the status bar.</p>
</div>
<div class="paragraph">
<p>The build will show a lot of errors while still working. Eventually the whole project should have been compiled without any errors. Unfortunately, due to a <a href="https://github.com/eclipse/n4js/issues/1373">known issue</a>, two problems exists. Please have a look at the linked issue on how to fix that (it is quite easy).</p>
</div>
<div class="sect4">
<h5 id="_changing-the-setup-script"><a class="anchor" href="#_changing-the-setup-script"></a><a class="link" href="#_changing-the-setup-script">2.2.1.1. Changing the Setup Script</a></h5>
<div class="paragraph">
<p>The setup scripts is stored at</p>
</div>
<div class="paragraph">
<p><code>n4js/releng/org.eclipse.n4js.targetplatform/N4JS.setup</code></p>
</div>
<div class="paragraph">
<p>Details about Oomph-Setup scripts can be found at</p>
</div>
<div class="paragraph">
<p><a href="https://wiki.eclipse.org/Eclipse_Installer" class="bare">https://wiki.eclipse.org/Eclipse_Installer</a></p>
</div>
</div>
</div>
<div class="sect3">
<h4 id="_manual-ide-configuration"><a class="anchor" href="#_manual-ide-configuration"></a><a class="link" href="#_manual-ide-configuration">2.2.2. Manual IDE Configuration</a></h4>
<div class="admonitionblock warning">
<table>
<tr>
<td class="icon">
<i class="fa icon-warning" title=""></i>
</td>
<td class="content">
Manual IDE configuration is not recommended!
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>For a manual install, clone the code and import all top-level projects from the docs, features, plugins, releng, testhelpers, and tests folders. Activate the targetplatform contained in the <code>releng/org.eclipse.n4js.targetplatform/</code> project.</p>
</div>
<div class="paragraph">
<p>The N4JS IDE is developed with Eclipse 2019-06 or better since the system is based on Eclipse anyway.
It is almost impossible to use another IDE to develop Eclipse plugins. The list of required plugins includes:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Xtext/Xtend 2.18.0</p>
</li>
<li>
<p>Xcore 1.9.0</p>
</li>
<li>
<p>Xpect 0.2.0.201906240918 from <a href="https://ci.eclipse.org/xpect/job/Xpect-Integration-Release/20/artifact/org.eclipse.xpect.releng/p2-repository/target/repository/" class="bare">https://ci.eclipse.org/xpect/job/Xpect-Integration-Release/20/artifact/org.eclipse.xpect.releng/p2-repository/target/repository/</a></p>
</li>
</ul>
</div>
<div class="paragraph">
<p>It is important to use the latest version of Xtext and the corresponding service release of Xcore. You will find the latest version numbers and plugins used in the target platform definition at
<a href="https://github.com/eclipse/n4js/blob/master/releng/org.eclipse.n4js.targetplatform/org.eclipse.n4js.targetplatform.target" class="bare">https://github.com/eclipse/n4js/blob/master/releng/org.eclipse.n4js.targetplatform/org.eclipse.n4js.targetplatform.target</a></p>
</div>
<div class="paragraph">
<p>You may need to adjust some settings in Eclipse, most importantly</p>
</div>
<div class="ulist">
<ul>
<li>
<p><strong>Text file encoding</strong> to <code>Other: UTF-8</code> and</p>
</li>
<li>
<p><strong>New text file line delimiter</strong> to <code>Unix</code> .</p>
</li>
</ul>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_release-engineering"><a class="anchor" href="#_release-engineering"></a><a class="link" href="#_release-engineering">3. Release Engineering</a></h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="_nightly-build-on-eclipse-infrastructure"><a class="anchor" href="#_nightly-build-on-eclipse-infrastructure"></a><a class="link" href="#_nightly-build-on-eclipse-infrastructure">3.1. Nightly build on Eclipse infrastructure</a></h3>
<div class="paragraph">
<p>The N4JS IDE, headless n4jsc.jar, and the N4JS update site is being built on the Eclipse Common Build
Infrastructure (CBI). For this purpose the N4JS project is using a dedicated Jenkins instance, referred
to as a "Jenkins Instance Per Project" (JIPP) in Eclipse CBI documentation. At this time, the N4JS
project&#8217;s JIPP is running on the "old" infrastructure, not yet using docker. This will be migrated
at a later point in time.</p>
</div>
<div class="paragraph">
<p>The N4JS JIPP is available at: <a href="https://ci.eclipse.org/n4js/" class="bare">https://ci.eclipse.org/n4js/</a></p>
</div>
<div class="paragraph">
<p>The nightly build performs the following main steps:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>compile the N4JS implementation,</p>
</li>
<li>
<p>build the n4jsc.jar, the IDE products for MacOS, Windows, Linux, and the update site,</p>
</li>
<li>
<p>run tests,</p>
</li>
<li>
<p>sign the IDE product for macOS and package it in a .dmg file,</p>
</li>
<li>
<p>deploy to n4jsc.jar, IDE products and update sites to Eclipse download server (i.e. download.eclipse.org),</p>
</li>
<li>
<p>move all artifacts older than 7 days from download.eclipse.org to archive.eclipse.org.</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>Details about all the above steps can be found in the Jenkinsfile <code>eclipse-nightly.jenkinsfile</code>, located in
the root folder of the N4JS source repository on GitHub.</p>
</div>
<div class="paragraph">
<p>The most accurate documentation for our JIPP can be found at <a href="https://wiki.eclipse.org/IT_Infrastructure_Doc" class="bare">https://wiki.eclipse.org/IT_Infrastructure_Doc</a>.
Note that many other documents do not apply to our JIPP, at the moment, as they refer to the new
infrastructure, e.g. <a href="https://wiki.eclipse.org/CBI" class="bare">https://wiki.eclipse.org/CBI</a> and <a href="https://wiki.eclipse.org/Jenkins" class="bare">https://wiki.eclipse.org/Jenkins</a>.</p>
</div>
</div>
<div class="sect2">
<h3 id="_build-the-n4js-ide-from-command-line"><a class="anchor" href="#_build-the-n4js-ide-from-command-line"></a><a class="link" href="#_build-the-n4js-ide-from-command-line">3.2. Build the N4JS IDE from command line</a></h3>
<div class="paragraph">
<p>Ensure you have</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Java 11</p>
</li>
<li>
<p>Maven 3.2.x and</p>
</li>
<li>
<p>Node.js 8</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>installed on your system.</p>
</div>
<div class="paragraph">
<p>Clone the repository</p>
</div>
<div class="listingblock">
<div class="content">
<pre>git clone https://github.com/Eclipse/n4js.git</pre>
</div>
</div>
<div class="paragraph">
<p>Change to the n4js folder:</p>
</div>
<div class="listingblock">
<div class="content">
<pre>cd n4js</pre>
</div>
</div>
<div class="paragraph">
<p>Run the Maven build:</p>
</div>
<div class="listingblock">
<div class="content">
<pre>mvn clean verify</pre>
</div>
</div>
<div class="paragraph">
<p>You may have to increase the memory for maven via <code>export MAVEN_OPTS="-Xmx2048m"</code> (Unix) or <code>set MAVEN_OPTS="-Xmx2048m"</code> (Windows).</p>
</div>
<div class="paragraph">
<p>Available optional maven profiles are:</p>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
buildProduct
</td>
<td class="hdlist2">
<p>create IDE products (Windows, macOS, Linux) and a jar for headless compilation</p>
</td>
</tr>
<tr>
<td class="hdlist1">
execute-plugin-tests
</td>
<td class="hdlist2">
<p>run OSGi tests (without UI)</p>
</td>
</tr>
<tr>
<td class="hdlist1">
execute-plugin-ui-tests
</td>
<td class="hdlist2">
<p>run UI-based OSGi tests</p>
</td>
</tr>
<tr>
<td class="hdlist1">
execute-ecmas-tests
</td>
<td class="hdlist2">
<p>run ECMA test suite</p>
</td>
</tr>
<tr>
<td class="hdlist1">
execute-smoke-tests
</td>
<td class="hdlist2">
<p>run generated tests using corrupted source code as input</p>
</td>
</tr>
<tr>
<td class="hdlist1">
execute-accesscontrol-tests
</td>
<td class="hdlist2">
<p>run generated tests for checking accessibility of class/interface members</p>
</td>
</tr>
<tr>
<td class="hdlist1">
execute-hlc-integration-tests
</td>
<td class="hdlist2">
<p>run integration tests using the headless jar (requires docker!)</p>
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>Available system properties:</p>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
noTests
</td>
<td class="hdlist2">
<p>suppress execution of all tests</p>
</td>
</tr>
<tr>
<td class="hdlist1">
startAndKeepVerdaccio
</td>
<td class="hdlist2">
<p>enforce starting and suppress stopping of the test verdaccio (see <a href="#sec:test-verdaccio">Test Verdaccio containing n4js-libs</a>)</p>
</td>
</tr>
</table>
</div>
<div class="sect3">
<h4 id="_publish-maven-tooling-code-org-eclipse-n4js-releng-util-code"><a class="anchor" href="#_publish-maven-tooling-code-org-eclipse-n4js-releng-util-code"></a><a class="link" href="#_publish-maven-tooling-code-org-eclipse-n4js-releng-util-code">3.2.1. Publish maven-tooling <code>org.eclipse.n4js.releng.util</code></a></h4>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title=""></i>
</td>
<td class="content">
For extending the N4JS-language in a different project, the <code>org.eclipse.n4js.releng.util</code> module needs to be published as a maven-plugin. You can deploy this SNAPSHOT-artifact to a local folder by providing the <code>local-snapshot-deploy-folder</code>-property pointing to an absolute path in the local file system:
</td>
</tr>
</table>
</div>
<div class="listingblock">
<div class="content">
<pre>mvn clean deploy -Dlocal-snapshot-deploy-folder=/var/lib/my/folder/local-mvn-deploy-repository</pre>
</div>
</div>
<div class="paragraph">
<p>The existence of <code>local-snapshot-deploy-folder</code> will trigger a profile enabling the deploy-goal for the project <code>org.eclipse.n4js.releng.util</code></p>
</div>
</div>
<div class="sect3">
<h4 id="sec:test-verdaccio"><a class="anchor" href="#sec:test-verdaccio"></a><a class="link" href="#sec:test-verdaccio">3.2.2. Test Verdaccio containing n4js-libs</a></h4>
<div class="paragraph">
<p>If profile <code>execute-hlc-integration-tests</code> is active, a local verdaccio instance is started and populated with
freshly-compiled n4js-libs (the libraries located under top-level folder <code>/n4js-libs</code>) and is stopped before the
end of the build. The verdaccio instance is started as a docker container called <code>n4js-test-verdaccio</code>.</p>
</div>
<div class="paragraph">
<p>When giving <code>-DstartAndKeepVerdaccio</code> on the command line, such a test verdaccio will always be started/populated but
never stopped, regardless of whether profile <code>execute-hlc-integration-tests</code> is active or not. This is useful to enforce
starting of the test verdaccio (even without running integration tests) and then reusing it in subsequent builds.</p>
</div>
</div>
<div class="sect3">
<h4 id="_generation-of-eclipse-help-for-spec-and-design-document"><a class="anchor" href="#_generation-of-eclipse-help-for-spec-and-design-document"></a><a class="link" href="#_generation-of-eclipse-help-for-spec-and-design-document">3.2.3. Generation of Eclipse help for spec and design document</a></h4>
<div class="paragraph">
<p>The HTML pages for N4JSSpec and N4JSDesign documents are generated from the Asciidoc sources in the project <code>org.eclipse.n4js.spec</code> <code>org.eclipse.n4js.design</code> by Asciispec. </p>
</div>
<div id="img:eclipse-help-doc-process" class="imageblock">
<div class="content">
<img src="chapters/03_releng/images/eclipse-help-process.svg" alt="Creating Eclipse help for N4JSSpec">
</div>
<div class="title">Figure 1. The process of creating Eclipse help for N4JSSpec</div>
</div>
<div class="paragraph">
<p>Figure <a href="#img:eclipse-help-doc-process">The process of creating Eclipse help for N4JSSpec</a> shows the generation process for N4JSSpec document. The process for N4JSDesign (and other adoc documents) is the same. The following explains the diagram.</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>Asciispec</code> is used to compile the source N4JSSpec Asciidoc into a single large <code>N4JSSpec.html</code> file which contains all the chapters. The use of the custom parameter <code>-a eclipse-help-mode</code> indicates that a special header and footer styles as well as CSS style should be used (i.e. no table of content menu, no download links etc.). Here, we are using the possibility provided by Asciidoctor to configure header/footer as well as CSS style via parameter <code>:docinfodir:</code> and <code>:stylesheet:</code>.</p>
</li>
<li>
<p>Our custom tool <code>Chunker</code> splits <code>N4JSSpec.html</code> (and other documents) into multiple chunked HTML files, each of which corresponds to either the <code>index</code> file or a chapter. It automatically re-writes internal links.</p>
</li>
<li>
<p>Another custom tool <code>EclipseHelpTOCGenerator</code> takes to Docbook file <code>N4JSSpec.xml</code> and generates an XML file describing the table of content (TOC) in the Eclipse format. This TOC file references the chunked HTML files above.</p>
</li>
<li>
<p>Another custom tool <code>IndexTocGenerator</code> takes to Docbook file <code>N4JSSpec.xml</code> similar to <code>EclipseHelpTOCGenerator</code>, but it generates an HTML fragment which can be embedded into the <code>index.html</code> page generated by the <code>Chunker</code> (Thus it has to run before the Chunker in that case).</p>
</li>
</ul>
</div>
</div>
</div>
<div class="sect2">
<h3 id="_updating-frameworks-and-dependencies"><a class="anchor" href="#_updating-frameworks-and-dependencies"></a><a class="link" href="#_updating-frameworks-and-dependencies">3.3. Updating frameworks and dependencies</a></h3>
<div class="sect3">
<h4 id="_update-of-eclipse-emf-xtext-etc"><a class="anchor" href="#_update-of-eclipse-emf-xtext-etc"></a><a class="link" href="#_update-of-eclipse-emf-xtext-etc">3.3.1. Update of Eclipse, EMF, Xtext, etc.</a></h4>
<div class="paragraph">
<p>For updating the N4JS IDE to a new version of Eclipse, EMF, Xtext, etc. follow these steps:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Create a new branch.</p>
</li>
<li>
<p>Bump versions of all dependencies mentioned in file <code>N4JS.setup</code>:</p>
<div class="olist loweralpha">
<ol class="loweralpha" type="a">
<li>
<p>Update all labels that refer to the version of the Ooomph setup (search for "label!" to find them).</p>
</li>
<li>
<p>Choose a new Eclipse version and define this in <code>N4JS.setup</code>.</p>
</li>
<li>
<p>For those other dependencies <em>that come with Eclipse</em> (e.g. EMF, Xtext) find out which version matches the chosen Eclipse version
and define that version in <code>N4JS.setup</code>.<br>
Tip: use the contents list of the SimRel you are targeting, e.g. <a href="https://projects.eclipse.org/releases/2019-03" class="bare">https://projects.eclipse.org/releases/2019-03</a></p>
</li>
<li>
<p>For those other dependencies <em>that are available via the Eclipse Orbit</em>, find out which version is the latest version available in
the Orbit and define that version in <code>N4JS.setup</code>.<br>
Tip: contents of the Eclipse Orbit can be found at <a href="https://download.eclipse.org/tools/orbit/downloads/" class="bare">https://download.eclipse.org/tools/orbit/downloads/</a><br>
(choose the correct link for the chosen Eclipse version!)</p>
</li>
<li>
<p>For all remaining dependencies (i.e. unrelated to Eclipse and not in Orbit), choose a version to use and define it in <code>N4JS.setup</code>.</p>
</li>
</ol>
</div>
</li>
<li>
<p>Check <code>Require-Bundle</code> sections of MANIFEST.MF files by searching for related bundle names or for <code>;bundle-version="</code>:</p>
<div class="olist loweralpha">
<ol class="loweralpha" type="a">
<li>
<p>There should be at most one version constraint for a specific bundle<br>
NOTE: the version constraints in the MANIFEST.MF files are just lower bounds and - at this time - we do not bump them to the latest version, in most cases.</p>
</li>
<li>
<p>There should be no version constraints to our bundles (i.e. <code>org.eclipse.n4js&#8230;&#8203;</code>)</p>
</li>
</ol>
</div>
</li>
<li>
<p>Review parent pom.xml files, i.e. <code>releng/org.eclipse.n4js.parent/pom.xml</code>:</p>
<div class="olist loweralpha">
<ol class="loweralpha" type="a">
<li>
<p>Update property <code>xtext-version</code>.</p>
</li>
<li>
<p>Check all other <code>*-version</code> properties and update them where needed.</p>
</li>
</ol>
</div>
</li>
<li>
<p>Update target platform file <code>org.eclipse.n4js.targetplatform.target</code> using Ooomph&#8217;s auto-generation:</p>
<div class="olist loweralpha">
<ol class="loweralpha" type="a">
<li>
<p>Start the Eclipse Installer.</p>
</li>
<li>
<p>Update the Eclipse Installer (using the button with the turning arrows).</p>
</li>
<li>
<p>On the second page, add the <code>N4JS.setup</code> file from your branch to the Eclipse Installer, using a GitHub raw(!) URL:<br>
<code><a href="https://raw.githubusercontent.com/eclipse/n4js/BRANCH_NAME/releng/org.eclipse.n4js.targetplatform/N4JS.setup" class="bare">https://raw.githubusercontent.com/eclipse/n4js/BRANCH_NAME/releng/org.eclipse.n4js.targetplatform/N4JS.setup</a></code></p>
</li>
<li>
<p>Ooomph a new development environment with this setup.</p>
</li>
<li>
<p>In the new Eclipse workspace created by Ooomph, the target platform file should have uncommitted changes:</p>
<div class="olist lowerroman">
<ol class="lowerroman" type="i">
<li>
<p>carefully review these changes, to be sure they make sense, and then</p>
</li>
<li>
<p>commit &amp; push those changes to your branch.</p>
</li>
</ol>
</div>
</li>
</ol>
</div>
</li>
<li>
<p>Thoroughly test the new versions, including some manual(!) tests:</p>
<div class="olist loweralpha">
<ol class="loweralpha" type="a">
<li>
<p>Run Jenkins builds.</p>
</li>
<li>
<p>Ooomph another N4JS development environment with Eclipse Installer.
This time, after Ooomphing is completed, the target platform file should no longer have any uncommitted changes.</p>
</li>
<li>
<p>Ensure the following types of tests can be executed locally in the newly installed Eclipse:</p>
<div class="olist lowerroman">
<ol class="lowerroman" type="i">
<li>
<p>plain JUnit tests (e.g. <code>org.eclipse.n4js.lang.tests</code>).</p>
</li>
<li>
<p>Plugin tests.</p>
</li>
<li>
<p>Plugin UI tests.</p>
</li>
<li>
<p>SWTBot tests.</p>
</li>
<li>
<p>Xpect tests (individual files and entire bundles; e.g. <code>org.eclipse.n4js.spec.tests</code>).</p>
</li>
<li>
<p>Xpect UI tests.</p>
</li>
</ol>
</div>
</li>
<li>
<p>Ensure an N4JS IDE product can be launched from within the newly installed Eclipse using the launch configuration
provided in the n4js repository.</p>
</li>
<li>
<p>After launching the N4JS IDE product, refresh the workspace and review/commit any changes in file <code>N4JS__IDE.launch</code>.</p>
</li>
<li>
<p>Download a product created in a Jenkins CI build and test it manually.</p>
</li>
<li>
<p>After merging to master: download a product created in a nightly build and test it manually.
Ensure signing and JRE bundling are still working properly.</p>
</li>
</ol>
</div>
</li>
</ol>
</div>
<div class="paragraph">
<p>All the above steps need to be performed in the <code>n4js-n4</code> repository, accordingly (e.g. file <code>N4JS-N4.setup</code>).</p>
</div>
</div>
<div class="sect3">
<h4 id="_update-of-the-embedded-jre"><a class="anchor" href="#_update-of-the-embedded-jre"></a><a class="link" href="#_update-of-the-embedded-jre">3.3.2. Update of the embedded JRE</a></h4>
<div class="paragraph">
<p>For updating the embedded JRE inside the N4JS IDE follow these steps:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Given a new JRE download location for Linux, MacOS and Windows with a common new version</p>
</li>
<li>
<p>Update the location related properties in the pom.xml files of</p>
<div class="olist loweralpha">
<ol class="loweralpha" type="a">
<li>
<p>n4js/builds/pom.xml</p>
</li>
<li>
<p>n4js/builds/org.eclipse.n4js.jre.linux.gtk.x86_64/pom.xml</p>
</li>
<li>
<p>n4js/builds/org.eclipse.n4js.jre.macosx.cocoa.x86_64/pom.xml</p>
</li>
<li>
<p>n4js/builds/org.eclipse.n4js.jre.win32.win32.x86_64/pom.xml</p>
</li>
</ol>
</div>
</li>
<li>
<p>Update the versions at all following locations:</p>
<div class="olist loweralpha">
<ol class="loweralpha" type="a">
<li>
<p>n4js/builds/org.eclipse.n4js.jre.linux.gtk.x86_64/META-INF/MANIFEST.MF</p>
</li>
<li>
<p>n4js/builds/org.eclipse.n4js.jre.linux.gtk.x86_64/META-INF/p2.inf</p>
</li>
<li>
<p>n4js/builds/org.eclipse.n4js.jre.macosx.cocoa.x86_64/META-INF/MANIFEST.MF</p>
</li>
<li>
<p>n4js/builds/org.eclipse.n4js.jre.macosx.cocoa.x86_64/META-INF/p2.inf</p>
</li>
<li>
<p>n4js/builds/org.eclipse.n4js.jre.win32.win32.x86_64/META-INF/MANIFEST.MF</p>
</li>
<li>
<p>n4js/builds/org.eclipse.n4js.jre.win32.win32.x86_64/META-INF/p2.inf</p>
</li>
</ol>
</div>
</li>
<li>
<p>Update the openjdk docker image used as base image in the "FROM" line at the top of all docker files:</p>
<div class="olist loweralpha">
<ol class="loweralpha" type="a">
<li>
<p>n4js-n4/jenkins/docker-build/Dockerfile</p>
</li>
</ol>
</div>
</li>
</ol>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_tips-and-tricks"><a class="anchor" href="#_tips-and-tricks"></a><a class="link" href="#_tips-and-tricks">4. Tips and Tricks</a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>In this chapter we collect some coding hints and guidelines on how to properly use the APIs of Eclipse, EMF, Xtext and
other dependencies we are using, as well as our own utilities and helpers.</p>
</div>
<div class="paragraph">
<p>This chapter is only about coding; add information on things like Eclipse setup or Maven/Jenkins to one of the preceding
chapters. Similarly, this chapter is intended to provide just a quick overview, check-list and reminder; add detailed
information and diagrams to one of the succeeding chapters.</p>
</div>
<div class="sect2">
<h3 id="_naming"><a class="anchor" href="#_naming"></a><a class="link" href="#_naming">4.1. Naming</a></h3>
<div class="ulist">
<ul>
<li>
<p>The internal handling of N4JS project names is non-trivial (due to the support for npm scopes), see
API documentation of <code>ProjectDescriptionUtils#isProjectNameWithScope(String)</code> for a detailed overview.
In short:</p>
<div class="ulist">
<ul>
<li>
<p><code>IN4JSProject#getProjectName()</code> and <code>IProject#getName()</code> return different values!</p>
</li>
<li>
<p>Avoid using the Eclipse project name, i.e. the return value of <code>IProject#getName()</code>, as far as possible
(only use it in UI code when actually dealing with what is shown in the Eclipse UI).</p>
</li>
<li>
<p>The last segment of an URI or path pointing to an N4JS project is <strong>not</strong> always the project name; use
utilities in <code>ProjectDescriptionUtils</code> instead, e.g. <code>#deriveN4JSProjectNameFromURI()</code>!
(However, given an URI or path pointing to a file inside an N4JS project, you can use its last segment
to obtain the file name.)</p>
</li>
</ul>
</div>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_logging"><a class="anchor" href="#_logging"></a><a class="link" href="#_logging">4.2. Logging</a></h3>
<div class="paragraph">
<p>In many situations developer needs to use some kind of logging. When in need, follow these rules:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Use <code>org.apache.log4j.Logger;</code> for logging. Other logging utilities (like java built in logger) are not configured.</p>
</li>
<li>
<p>do not use <code>System.out</code> nor <code>Sysetem.err</code> for logging. It is ok to use it for debugging purposes, but those calls
should never be merged to master. <em>(with exception of headless compiler, which uses them explicitly)</em></p>
</li>
<li>
<p>There is central logger configuration in <code>org.eclipse.n4js.utils.logging</code> (and <code>org.eclipse.n4js.utils.logging</code>) that should
be used</p>
<div class="olist loweralpha">
<ol class="loweralpha" type="a">
<li>
<p><code>log4j.xml</code> used for production</p>
</li>
<li>
<p><code>log4j_tests.xml</code> used when running tests</p>
</li>
</ol>
</div>
</li>
<li>
<p>in Eclipse run configurations logger has to be set properly, e.g.
<code>log4j.configuration=file:${workspace_loc:org.eclipse.n4js.utils.logging/log4j_tests.xml}</code></p>
</li>
<li>
<p>in maven configurations logger has to be set separately, e.g.
<code>-Dlog4j.configuration="file:${basedir}/../../plugins/org.eclipse.n4js.utils.logging/log4j_tests.xml</code></p>
</li>
</ol>
</div>
</div>
<div class="sect2">
<h3 id="_cancellation-handling"><a class="anchor" href="#_cancellation-handling"></a><a class="link" href="#_cancellation-handling">4.3. Cancellation Handling</a></h3>
<div class="paragraph">
<p>At various occasions, Xtext provides an instance of class <code>CancelIndicator</code> to allow our code to handle cancellation of
long-running task.</p>
</div>
<div class="paragraph">
<p>Some things to keep in mind:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>whenever a <code>CancelIndicator</code> is available any code that might not return immediately should implement proper
cancellation handling (as explained in the next items).</p>
</li>
<li>
<p>most importantly: reacting to a cancellation by returning early from a method is an anti-pattern that leads to
problems (client code might continue work on a canceled and thus invalid state); instead: throw an
<code>OperationCanceledException</code>!</p>
</li>
<li>
<p>don&#8217;t use <code>CancelIndicator#isCanceled()</code> for cancellation handling, except in certain special cases. A valid exception
case might be during logging to show a message like "operation was canceled".</p>
</li>
<li>
<p>instead, inject the Xtext service called <code>OperationCanceledManager</code> and invoke its method <code>#checkCanceled()</code>, passing-in
the cancel indicator (this method is null-safe; it will throw an <code>OperationCanceledException</code> in case a cancellation has
occurred). Don&#8217;t directly create and throw an <code>OperationCanceledException</code> yourself.</p>
</li>
<li>
<p>use the other methods provided by <code>OperationCanceledManager</code> when appropriate (see code of that class for details).</p>
</li>
<li>
<p>in try/catch blocks, when catching exceptions of a super type of <code>OperationCanceledException</code>, be sure to <strong>not suppress</strong>
cancellation exceptions. For example:</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-java" data-lang="java">// Java code
@Inject private OperationCanceledManager operationCanceledManager;
/** Returns true on success, false otherwise. */
public boolean doSomething(CancelIndicator ci) {
try {
// do something that might be canceled
return true;
} catch(Exception e) {
operationCanceledManager.propagateIfCancelException(e); // &lt;- IMPORTANT!
return false;
}
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>Try/finally blocks, on the other hand, do not need any special handling.</p>
</div>
</li>
<li>
<p>a cancel indicator can also be stored in the rule environment (see <code>RuleEnvironmentExtensions#addCancelIndicator()</code>). This
means:</p>
<div class="ulist">
<ul>
<li>
<p>if you create a rule environment completely from scratch and you have a cancel indicator at hand, add it to the rule
environment via <code>RuleEnvironmentExtensions#addCancelIndicator()</code> (not required when using <code>RuleEnvironmentExtensions#wrap()</code> for
deriving a rule environment from an existing one).</p>
</li>
<li>
<p>if you have a rule environment available, be sure to use its cancel indicator in long-running operations, i.e. with
code like:</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-java" data-lang="java">// Xtend code
import static extension org.eclipse.n4js.typesystem.utils.RuleEnvironmentExtensions.*
class C {
@Inject private OperationCanceledManager operationCanceledManager;
def void doSomething() {
for(a : aLotOfStuff) {
operationCanceledManager.checkCanceled(G.cancelIndicator);
// main work ...
}
}</code></pre>
</div>
</div>
</li>
</ul>
</div>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_caching"><a class="anchor" href="#_caching"></a><a class="link" href="#_caching">4.4. Caching</a></h3>
<div class="ulist">
<ul>
<li>
<p>Caching of external libraries (implemented in ExternalProjectMappings)</p>
<div class="ulist">
<ul>
<li>
<p>update <em>only</em> using <code>EclipseExternalLibraryWorkspace#updateState()</code></p>
</li>
<li>
<p>always mind that the diff of current state and cached state is a necessary information for cleaning dependencies of removed npms</p>
<div class="ulist">
<ul>
<li>
<p>see <code>EclipseExternalIndexSynchronizer#synchronizeNpms()</code> for implementation</p>
</li>
</ul>
</div>
</li>
<li>
<p>updating also happens when external root locations change (see ExternalIndexUpdater)</p>
</li>
</ul>
</div>
</li>
<li>
<p>Caching of user workspace projects (implemented in MuliCleartriggerCache)</p>
<div class="ulist">
<ul>
<li>
<p>caches only some project information and should be refactored along with Core, Model and EclipseBasedN4JSWorkspace</p>
</li>
</ul>
</div>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_dependency-injection"><a class="anchor" href="#_dependency-injection"></a><a class="link" href="#_dependency-injection">4.5. Dependency Injection</a></h3>
<div class="paragraph">
<p>There are some things to keep in mind when using dependency injection in the context of Xtext. This is a longer topic and it is discussed in the appendix
<a href="#sec:XtextInjection">Xtext Injection</a>.</p>
</div>
</div>
<div class="sect2">
<h3 id="_miscellaneous"><a class="anchor" href="#_miscellaneous"></a><a class="link" href="#_miscellaneous">4.6. Miscellaneous</a></h3>
<div class="ulist">
<ul>
<li>
<p>Resource load states: when an N4JS/N4JSD file is loaded, a certain sequence of processing is triggered (parsing,
linking, validation, etc.) and thus an <code>N4JSResource</code> transitions through a sequence of "load states". For details,
see <a href="#sec:N4JS_Resource_Load_States">N4JS Resource Load States</a>.</p>
</li>
</ul>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_parser"><a class="anchor" href="#_parser"></a><a class="link" href="#_parser">5. Parser</a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>Some of the concepts described here were presented at
<a href="https://www.youtube.com/watch?v=Xm-7aE1UMGY&amp;feature=youtu.be">EclipseCon 2013</a> and
<a href="https://vimeo.com/channels/xtextcon/98446435">XtextCon 2014</a>. Note that the material presented at the linked videos may be outdated.</p>
</div>
<div class="sect2">
<h3 id="sec:Parser_Overview"><a class="anchor" href="#sec:Parser_Overview"></a><a class="link" href="#sec:Parser_Overview">5.1. Overview</a></h3>
<div class="paragraph">
<p>The parser is created from an Xtext grammar. Actually, there are several grammars used as shown in <a href="#fig:cd_grammars">Figure CD Grammars</a>. These grammars and the parsers generated from them are described more closely in the following sections.</p>
</div>
<div id="fig:cd_grammars" class="imageblock center">
<div class="content">
<img src="chapters/05_parser/images/cd_grammars.svg" alt="cd grammars">
</div>
<div class="title">Figure 2. N4 Grammars</div>
</div>
</div>
<div class="sect2">
<h3 id="sec:N4JS_Parser"><a class="anchor" href="#sec:N4JS_Parser"></a><a class="link" href="#sec:N4JS_Parser">5.2. N4JS Parser</a></h3>
<div class="paragraph">
<p>One of the most tricky parts of JavaScript is the parsing because there is a conceptual mismatch between the <a href="#AC">ANTLR</a> runtime and the specified grammar. Another challenge is the disambiguation of regular expressions and binary operations. Both features require significant customizing of the generated parser (see figure below).</p>
</div>
<div id="fig:cd_ASIParser" class="imageblock center">
<div class="content">
<img src="chapters/05_parser/images/cd_ASIParser.svg" alt="cd ASIParser">
</div>
<div class="title">Figure 3. Overview custom parser implementation (runtime only)</div>
</div>
</div>
<div class="sect2 language-bash">
<h3 id="sec:Parser_Generation_Post_Processing"><a class="anchor" href="#sec:Parser_Generation_Post_Processing"></a><a class="link" href="#sec:Parser_Generation_Post_Processing">5.3. Parser Generation Post-Processing</a></h3>
<div class="paragraph">
<p>The ANTLR grammar that is generated by Xtext is post-processed to inject custom code into the grammar file before it is passed to the ANTLR tool. This is required in particular due to <a href="#AC">ASI</a> (Automated Semicolon Insertion), but for some other reasons as well.</p>
</div>
<div class="paragraph">
<p>Actually, there are several injections:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Due to Xtext restrictions, the generated ANTLR grammar file (<strong>*.g</strong>) is modified. This means that some some additional actions are added and some rules are rewritten.</p>
</li>
<li>
<p>Due to ANTLR restrictions, the generated ANTLR Java parser (<strong>*.java</strong>) os modified. This means that some generated rules are slightly modified to match certain requirements.</p>
</li>
<li>
<p>Due to Java restrictions, the generated Java parser needs to be preprocessed in order to reduce the size of certain methods since they must not exceed 64k characters. This is implemented by means of an MWE fragment, activated after the other post processing steps are done.</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>The first two steps are handled by <code>AntlrGeneratorWithCustomKeywordLogic</code>, which is configured with additional helpers in <code>GenerateN4JS.mwe2</code>. shows the customized classes which modify the code generation. These classes are all part of the <code>releng.utils</code> bundle.</p>
</div>
<div class="imageblock center">
<div class="content">
<img src="chapters/05_parser/images/cd_parsergeneration.svg" alt="cd parsergeneration">
</div>
<div class="title">Figure 4. Class Diagram Parser Generation</div>
</div>
<div class="sect3">
<h4 id="sec:Automatic_Semicolon_Insertion"><a class="anchor" href="#sec:Automatic_Semicolon_Insertion"></a><a class="link" href="#sec:Automatic_Semicolon_Insertion">5.3.1. Automatic Semicolon Insertion</a></h4>
<div class="paragraph">
<p>The EcmaScript specification mandates that valid implementations automatically insert a semicolon as a statement delimiter if it is missing and the input file would become invalid due to the missing semicolon. This is known as <a href="#AC">ASI</a>. It implies that not only valid implementations have to perform this, but a valid parser has to mimic this behavior in order to parse executable code. The <a href="#AC">ASI</a> is implemented by two different means.</p>
</div>
<div class="paragraph">
<p>The parser’s error recovery strategy is customized so it attempts to insert a semicolon if it was expected. Both strategies have to work hand in hand in order to consume all sorts of legal JavaScript code.</p>
</div>
<div class="sect4">
<h5 id="sec:Injected_code_in_the_Antlr_grammar_file"><a class="anchor" href="#sec:Injected_code_in_the_Antlr_grammar_file"></a><a class="link" href="#sec:Injected_code_in_the_Antlr_grammar_file">5.3.1.1. Injected code in the Antlr grammar file</a></h5>
<div class="paragraph">
<p>Under certain circumstances, the parser has to actively promote a token to become a semicolon even though it may be a syntactically a closing brace or line break. This has to happen before that token is consumed thus the rules for return statements, continue statements and break statements are enhanced to actively promote these tokens to semicolons.</p>
</div>
<div class="paragraph">
<p>The same rule is applied to promote line breaks between an expression and a possible postfix operator <code>++</code> or <code>–</code>. At this location the line break is always treated as a semicolon even though the operator may be validly consumed and produce a postfix expression.</p>
</div>
<div class="paragraph">
<p>In both cases, the method <code>promoteEOL()</code> is used to move a token that may serve as an automatically injected semicolon from the so called hidden token channel to the semantic channel. The hidden tokens are usually not handled by the parser explicitly thus they are semantically invisible (therefore the term hidden token). Nevertheless, they can be put on the semantic channel explicitly to make them recognizable. That’s implemented in the EOL promotion. The offending tokens include the hidden line terminators and multi-line comments that include line breaks. Furthermore, closing braces (right curly brackets) are included in the set of offending tokens as well as explicit semicolons.</p>
</div>
</div>
<div class="sect4">
<h5 id="sec:Customized_error_recovery"><a class="anchor" href="#sec:Customized_error_recovery"></a><a class="link" href="#sec:Customized_error_recovery">5.3.1.2. Customized error recovery</a></h5>
<div class="paragraph">
<p>Since the EOL promotion does not work well with Antlr prediction mode, another customization complements that feature. As soon as an invalid token sequence is attempted to be parsed and missing semicolon would make that sequence valid, an offending token is sought and moved to the semantic channel. This is implemented in the custom recovery strategy.</p>
</div>
</div>
</div>
<div class="sect3">
<h4 id="sec:_No_line_terminator_allowed_here__handling"><a class="anchor" href="#sec:_No_line_terminator_allowed_here__handling"></a><a class="link" href="#sec:_No_line_terminator_allowed_here__handling">5.3.2. Async and <code>No line terminator allowed here</code> Handling</a></h4>
<div class="paragraph">
<p>There is no way of directly defining <code>No line terminator allowed here</code>. This is required not only for <a href="#AC">ASI</a>, but also for <code>async</code>. This requires not only a special rule (using some rules from <a href="#sec:Automatic_Semicolon_Insertion">ASI</a>), but also a special error recovery since the token ’async’ may be rejected (by the manually enriched rule) which is of course unexpected behavior from the generated source code.</p>
</div>
</div>
<div class="sect3">
<h4 id="sec:Regular_Expression"><a class="anchor" href="#sec:Regular_Expression"></a><a class="link" href="#sec:Regular_Expression">5.3.3. Regular Expression</a></h4>
<div class="paragraph">
<p>The ANTLR parsing process can basically be divided into three steps. First of all, the file contents has to be read from disk. This includes the proper encoding of bytes to characters. The second step is the lexing or tokenizing of the character stream. A token is a basically a typed region in the stream, that is a triplet of token-id, offset and length. The last step is the parsing of these tokens. The result is a semantic model that is associated with a node tree. All necessary information to validate the model can be deduced from these two interlinked representations.</p>
</div>
<div class="imageblock center">
<div class="content">
<img src="chapters/05_parser/images/ad_parsing_simplified.svg" alt="ad parsing simplified">
</div>
<div class="title">Figure 5. Simplified visualization of the parsing</div>
</div>
<div class="paragraph">
<p>Since the default semantics and control flow of Antlr generated parsers do not really fit the requirements of a fully working JavaScript parser, some customizations are necessary. <strong>Regular expression literals in JavaScript cannot be syntactically disambiguated from div operations without contextual information.</strong> Nevertheless, the spec clearly describes, where a regular expression may appear and where it is prohibited. Unfortunately, it is not possible to implement these rules in the lexer alone, since it does not have enough contextual information. Therefore, the parser has been enhanced to establish a communication channel with the lexer. It announces when it expects a regular expression rather than a binary operation.</p>
</div>
<div class="paragraph">
<p>This required a reworking of the Antlr internals. Instead of a completely pre-populated <code>TokenStream</code>, the parser works on a lazy implementation that only reads as many characters as possible without a disambiguation between regular expression literals and divide operators.</p>
</div>
<div class="paragraph">
<p>Only after the parser has read this buffered tokens and potentially announced that it expects a regular expression, another batch of characters is processed by the lexer until the next ambiguous situation occurs. This is fundamentally different from the default behavior of Antlr.</p>
</div>
<div class="imageblock center">
<div class="content">
<img src="chapters/05_parser/images/sd_parsing_sequence.svg" alt="sd parsing sequence">
</div>
<div class="title">Figure 6. Abstract control and object flow during parsing</div>
</div>
<div class="paragraph">
<p>shows the involved classes which allow for this lexer-parser communication.</p>
</div>
<div class="imageblock center">
<div class="content">
<img src="chapters/05_parser/images/cd_parserlexercommunication.svg" alt="cd parserlexercommunication">
</div>
<div class="title">Figure 7. Class Diagram Parser-Lexer Communication</div>
</div>
</div>
<div class="sect3">
<h4 id="sec:Unicode"><a class="anchor" href="#sec:Unicode"></a><a class="link" href="#sec:Unicode">5.3.4. Unicode</a></h4>
<div class="paragraph">
<p>Unicode support in JavaScript includes the possibility to use unicode escape sequences in identifiers, string literals and regular expression literals. Another issue in this field is the specification of valid identifiers in JavaScript. They are described by means of unicode character classes. These have to be enumerated in the terminal rules in order to fully accept or reject valid or invalid JS identifiers.</p>
</div>
<div class="paragraph">
<p>For that purpose, a small code generator is used to define the terminal fragments for certain unicode categories. The <code>UnicodeGrammarGenerator</code> basically iterates all characters from <code>Character.MIN_VALUE</code> to <code>Character.MAX_VALUE</code> and adds them as alternatives to the respective terminal fragments, e.g. <code>UNICODE_DIGIT_FRAGMENT</code>.</p>
</div>
<div class="paragraph">
<p>The real terminal rules are defined as a composition of these generated fragments. Besides that, each character in an identifier, in a string literal or in a regular expression literal may be represented by its unicode escape value, e.g. ` u0060`. These escape sequences are handled and validated by the <code>IValueConverter</code> for the corresponding terminal rules.</p>
</div>
<div class="paragraph">
<p>The second piece of the puzzle are the unicode escaped sequences that may be used in keywords. This issue is covered by the <code>UnicodeKeywordHelper</code> which replaces the default terminal representation in the generated Antlr grammar by more elaborated alternatives. The keyword <code>if</code> is not only lexed as <code>’if’</code> but as seen in snippet
<a href="#lst:terminal_if">Terminal if listing</a>.</p>
</div>
<div id="lst:terminal_if" class="listingblock">
<div class="title">Terminal if</div>
<div class="content">
<pre class="highlight"><code>If :
( 'i' | '\\' 'u' '0`` 0`` 6`` 9' )
( 'f' | '\\' 'u' '0`` 0`` 6`` 6' );</code></pre>
</div>
</div>
</div>
<div class="sect3">
<h4 id="sec:Literals"><a class="anchor" href="#sec:Literals"></a><a class="link" href="#sec:Literals">5.3.5. Literals</a></h4>
<div class="paragraph">
<p>Template literals are also to be handled specially, see <code>TemplateLiteralDisambiguationInjector</code> for details.</p>
</div>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Modifiers"><a class="anchor" href="#sec:Modifiers"></a><a class="link" href="#sec:Modifiers">5.4. Modifiers</a></h3>
<div class="paragraph">
<p>On the AST side, all modifiers are included in a single enumeration <code>N4Modifier</code>. In the types model however, the individual modifiers are mapped to two different enumerations of <em>access</em> modifiers (namely <code>TypeAccessModifier</code> and <code>MemberAccessModifier</code>) and a number of boolean properties (in case of non-access modifiers such as <code>abstract</code> or <code>static</code>). This mapping is done by the types builder, mostly by calling methods in class <code>ModifierUtils</code>.</p>
</div>
<div class="paragraph">
<p>The grammar allows the use of certain modifiers in many places that are actually invalid. Rules where a certain modifier may appear in the AST are implemented in method isValid(EClass,N4Modifier) in class <code>ModifierUtils</code> and checked via several validations in <code>N4JSSyntaxValidator</code>. Those validations also check for a particular order of modifiers that is not enforced by the grammar.</p>
</div>
<div class="paragraph">
<p>See API documentation of enumeration <code>N4Modifier</code> in file <code>N4JS.xcore</code> and the utility class <code>ModifierUtils</code> for more details.</p>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Conflict_Resolutions"><a class="anchor" href="#sec:Conflict_Resolutions"></a><a class="link" href="#sec:Conflict_Resolutions">5.5. Conflict Resolutions</a></h3>
<div class="sect3">
<h4 id="sec:Reserved_Keywords_vs__Identifier_Names"><a class="anchor" href="#sec:Reserved_Keywords_vs__Identifier_Names"></a><a class="link" href="#sec:Reserved_Keywords_vs__Identifier_Names">5.5.1. Reserved Keywords vs. Identifier Names</a></h4>
<div class="paragraph">
<p>Keywords and identifiers have to be distinguished by the lexer. Therefore, there is no means to decide upfront whether a certain keyword is actually used as a keyword or whether it is used as an identifier in a given context. This limitation is idiomatically overcome by a data type rule for valid identifiers. This data type rule enumerates all keywords which may be used as identifiers and the pure IDENTIFIER terminal rule as seen in <a href="#lst:keywords_as_identifier">Keywords as Identifier listing</a>.</p>
</div>
<div id="lst:keywords_as_identifier" class="listingblock">
<div class="title">Keywords as Identifier</div>
<div class="content">
<pre class="highlight"><code class="language-ebnf" data-lang="ebnf">N4JSIdentifier: IDENTIFIER
| 'get'
| 'set'
...
;</code></pre>
</div>
</div>
</div>
<div class="sect3">
<h4 id="sec:Operators_and_Generics"><a class="anchor" href="#sec:Operators_and_Generics"></a><a class="link" href="#sec:Operators_and_Generics">5.5.2. Operators and Generics</a></h4>
<div class="paragraph">
<p>The ambiguity between shift operators and nested generics arises also from the fact, that Antlr lexer upfront without any contextual information. When implemented naively, the grammar will be broken, since a token sequence <code>a&gt;&gt;b</code> can either be part of <code>List&lt;List&lt;a&gt;&gt; b</code> or it can be part of a binary operation <code>int c = a &gt;&gt; b</code>. Therefore the shift operator may not be defined with a single token but has to be composed from individual characters (see <a href="#lst:shift_operator">Shift Operator listing</a>).</p>
</div>
<div id="lst:shift_operator" class="listingblock">
<div class="title">Shift Operator listing</div>
<div class="content">
<pre class="highlight"><code class="language-ebnf" data-lang="ebnf">ShiftOperator:
'&gt;' '&gt;' '&gt;'?
| '&lt;' '&lt;'
;</code></pre>
</div>
</div>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Content_Assist_Parser"><a class="anchor" href="#sec:Content_Assist_Parser"></a><a class="link" href="#sec:Content_Assist_Parser">5.6. Content-Assist Parser</a></h3>
<div class="admonitionblock warning">
<table>
<tr>
<td class="icon">
<i class="fa icon-warning" title=""></i>
</td>
<td class="content">
This section may be outdated!
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>The <a href="#AC">CA</a> parser also needs adjustments for supporting automatic semicolon insertion and regular expressions. Instead of modifying the <a href="#AC">CA</a> parser generator similar to the normal parser, the former reuses parts of the latter as far as possible. That is, the token sequence that is produced during production parsing is used as is for the content assist parser. Semicolons have already been inserted where appropriate and regular expression are successfully distinguished from divide operators.</p>
</div>
<div class="paragraph">
<p>Since the n4js grammar uses syntactic predicates, the content assist parser is compiled with backtracking enabled. This is always the case for Xtext’s CA parsers that rely on backtracking or predicates (local backtracking) in the production parser. This approach is both good (CA works in general) and bad (unpredictable decisions in case of error at locations prior to the cursor). Since parsing with backtracking enabled makes for a fundamental difference in how the prediction and parsing works and how the parser decides which decision paths to take, the customization patterns from the production parser are not applied 1:1 to the CA parser, but adapted instead. The content assist parser doesn’t use a freshly lexed token stream with unicode support, ASI or regular expression literals, but instead uses a synthesized token sequence which is rebuilt from the existing node model.</p>
</div>
<div class="paragraph">
<p>The token stream that is consumed by the content assist parser is therefore not created by a lexer but by the <code>org.eclipse.n4js.ui.contentassist.NodeModelTokenSource</code>.
It traverses the existing node model that is contained in the resource and was produced by the production parser. This approach has the significant advantage that any decision that was made by that parser is also immediately applicable to the content assist infrastructure. For that purpose, the leaf nodes of the node model are mapped to ANTLR token types.
This is achieved by the <code>org.eclipse.n4js.ui.contentassist.ContentAssistTokenTypeMapper</code> which is capable to provide the untyped ANTLR token type (primitive int) for a given grammar element.</p>
</div>
<div class="paragraph">
<p>Special considerations have been made for the last token in the produced source. If it overlaps with an existing leaf node but does not fully cover it, the plain Antlr lexer is used to consume the prefix that is overlapping. Since the terminals will never overlap with each other the longest match always wins without backtracking in the lexer, it is save to assume that only one token is produced from the prefix. The very last token in the <code>org.eclipse.n4js.ui.contentassist.NodeModelTokenSource</code> is always the EOF token (<code>org.antlr.runtime.Token.EOF_TOKEN</code>).</p>
</div>
<div class="paragraph">
<p>Given that the token source is equal to the prefix in the production token source, some more thought has to be put into the synthesized end of file. The production parser used the complete file to decide where to automatically insert a semicolon and where not to. This would potentially change if there was another token next to the artificial EOF. Therefore, two cases have to considered. The first one describes CA request next to an automatically inserted semicolon and the second one describes CA requests at a position where a semicolon could have been inserted if the token to the right was another one. The <code>org.eclipse.n4js.ui.contentassist.CustomN4JSParser</code> reflects these cases. Heuristics are applied to the end of the token sequence to decide whether a second pass has to be performed to collect yet more following elements. Based on the concrete sequence, the last automatically inserted semicolon is removed from the sequence prior to the second pass or such is a token is explicitly synthesized and appended. Besides the second pass, another special treatment is made for postfix expressions. Those may not be interrupted by a hidden semicolon so those are filtered from the resulting follow set if appropriate.</p>
</div>
<div class="paragraph">
<p>The parser is used by the <code>org.eclipse.n4js.ui.contentassist.ContentAssistContextFactory</code> where all relevant entry points from the super class are specialized to pass the node model in the the parser facade (<code>org.eclipse.n4js.ui.contentassist.CustomN4JSParser</code>). In that sense, the ContentAssistContextFactory serves as a drop-in replacement binding the default <code>ParserBasedContentAssistContextFactory.StatefulFactory</code>.</p>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_type-system"><a class="anchor" href="#_type-system"></a><a class="link" href="#_type-system">6. Type System</a></h2>
<div class="sectionbody">
<div class="sect2 language-n4js">
<h3 id="sec:Type_Model_and_Grammar"><a class="anchor" href="#sec:Type_Model_and_Grammar"></a><a class="link" href="#sec:Type_Model_and_Grammar">6.1. Type Model and Grammar</a></h3>
<div class="paragraph">
<p>The type model is used to define actual types and their relations (meta-model is defined by means of Xcore in file <code>Types.xcore</code>)
and also references to types (meta-model in <code>TypeRefs.xcore</code>). The type model is built via the <code>N4JSTypesBuilder</code> when a resource
is loaded and processed, and most type related tasks work only on the type model. Some types that are (internally) available
in N4JS are not defined in N4JS, but instead in a special, internal type language not available to N4JS developers, called N4TS
and defined in file <code>Types.xtext</code>.</p>
</div>
<div class="paragraph">
<p>The types are referenced by AST elements; vice versa the AST elements can be referenced from the types (see <code>SyntaxRelatedTElement</code>).
This backward reference is a simple reference to an EObject.</p>
</div>
<div class="sect3">
<h4 id="sec:Type_Model_Overview"><a class="anchor" href="#sec:Type_Model_Overview"></a><a class="link" href="#sec:Type_Model_Overview">6.1.1. Type Model Overview</a></h4>
<div class="paragraph">
<p>The following figure, <a href="#fig:cd_typeAndTypeRefHierarchy">Types and Type References</a>, shows the classes of the type model and their inheritance relations, both the actual type definitions as defined in <code>Types.xcore</code> and the type references defined in <code>TypeRefs.xcore</code>. The most important type reference is the <code>ParameterizedTypeRef</code>; it is used for most user-defined references, for both parameterized and non-parameterized references. In the latter case, the list of type arguments is empty.</p>
</div>
<div id="fig:cd_typeAndTypeRefHierarchy" class="imageblock">
<div class="content">
<img src="chapters/06_typesystem/images/cd_typeModelHierarchy_allInOne.png" alt="cd typeModelHierarchy allInOne">
</div>
<div class="title">Figure 8. Type Model Overview: Types in the upper half and Type References in the lower half.</div>
</div>
<div class="paragraph">
<p>Most types are self-explanatory. <code>TypeDefs</code> is the container element used in N4TS. Note that not all types and properties of types are available in N4JS – some can only be used in the N4TS language or be inferred by the type system for internal purposes. Some types need some explanation:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>TObjectPrototype</code>: Metatype for defining built-in object types such as <code>Object</code> or <code>Date</code>, only available in N4TS.</p>
</li>
<li>
<p><code>VirtualBaseType</code>: This type is not available in N4JS. It is used to define common properties provided by all types of a certain metatype. E.g., it is used for defining some properties shared by all enumerations (this was the reason for introducing this type).</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>We distinguish four kinds of types as summarized in <a href="#tab:KindOfTypes">Kind Of Types</a>. Role is an internal construct for different kind of users who can define the special kind of type. The language column refers to the language used to specify the type; which is either N4JS or N4TS.</p>
</div>
<table id="tab:KindOfTypes" class="tableblock frame-all grid-all spread">
<caption class="title">Table 1. Kind of Types</caption>
<colgroup>
<col style="width: 10%;">
<col style="width: 10%;">
<col style="width: 10%;">
<col style="width: 70%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Kind</th>
<th class="tableblock halign-left valign-top">Language</th>
<th class="tableblock halign-left valign-top">Role</th>
<th class="tableblock halign-left valign-top">Remark</th>
</tr>
</thead>
<tbody>
<tr>
<th class="tableblock halign-left valign-top"><p class="tableblock">user</p></th>
<th class="tableblock halign-left valign-top"><p class="tableblock">N4JS</p></th>
<th class="tableblock halign-left valign-top"><p class="tableblock">developer</p></th>
<td class="tableblock halign-left valign-top"><p class="tableblock">User defined types, such as declared classes or functions. These types are to be explicitly defined or imported in the code.</p></td>
</tr>
<tr>
<th class="tableblock halign-left valign-top"><p class="tableblock">library</p></th>
<th class="tableblock halign-left valign-top"><p class="tableblock">N4JSD</p></th>
<th class="tableblock halign-left valign-top"><p class="tableblock">developer</p></th>
<td class="tableblock halign-left valign-top"><p class="tableblock">Type declarations only, comparable to C header files, without implementation. Used for defining the API of 3rd party libraries. These type definitions are to be explicitly defined or imported in the code.</p></td>
</tr>
<tr>
<th class="tableblock halign-left valign-top"><p class="tableblock">builtin</p></th>
<th class="tableblock halign-left valign-top"><p class="tableblock">N4TS</p></th>
<th class="tableblock halign-left valign-top"><p class="tableblock">smith</p></th>
<td class="tableblock halign-left valign-top"><p class="tableblock">Built-in ECMAScript objects interpreted as types. E.g., <code>String</code>, <code>Date</code>, <code>Math</code>. These types are provided by N4JS and are always available.</p></td>
</tr>
<tr>
<th class="tableblock halign-left valign-top"><p class="tableblock">primitive</p></th>
<th class="tableblock halign-left valign-top"><p class="tableblock">N4TS</p></th>
<th class="tableblock halign-left valign-top"><p class="tableblock">smith</p></th>
<td class="tableblock halign-left valign-top"><p class="tableblock">Primitive ECMAScript (and N4JS ties), such as <code>string</code>, <code>number</code>, <code>pathselector&lt;T&gt;</code>, <code>i18n</code>, and also <code>any</code>, <code>undefined</code> and <code>void</code>. These types are provided by N4JS and are always available. Primitive types are described in detail in the spec (see chapter "Primitive ECMAScript Types").</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect3">
<h4 id="sec:Built_in_Types"><a class="anchor" href="#sec:Built_in_Types"></a><a class="link" href="#sec:Built_in_Types">6.1.2. Built-in and Primitive Types</a></h4>
<div class="paragraph">
<p>The built-in and primitive types are not defined by the user, i.e. N4JS programmer. Instead, they are defined in special
internal files using the internal N4TS language: <code>builtin_js.n4ts</code>, <code>builtin_n4.n4ts</code>, <code>primitives_js.n4ts</code>, <code>primitives_n4.n4ts</code>.</p>
</div>
</div>
<div class="sect3 language-n4js">
<h4 id="sec:Type_Model_DSL"><a class="anchor" href="#sec:Type_Model_DSL"></a><a class="link" href="#sec:Type_Model_DSL">6.1.3. Type Model DSL</a></h4>
<div class="paragraph">
<p>For defining built-in types and for tests, a special DSL called N4TS is provided by means of an Xtext grammar and generated
tools. The syntax is similar to n4js, but of course without method or function bodies, i.e. without any statements of expressions.
The grammar file is found in <code>Types.xtext</code>.</p>
</div>
<div class="paragraph">
<p>The following list documents some differences to N4JS:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>access modifiers directly support <code>Internal</code>, so no annotations are needed (nor supported) here.</p>
</li>
<li>
<p>besides N4 classifiers such as classes, the following classifiers can be defined:</p>
<div class="olist loweralpha">
<ol class="loweralpha" type="a">
<li>
<p><code>object</code> define classes derived from object (predefined object types) Special features:</p>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
indexed
</td>
<td class="hdlist2">
<p>defined what type is returned in case of index access</p>
</td>
</tr>
</table>
</div>
</li>
<li>
<p><code>virtualBase</code> virtual base types for argument</p>
</li>
</ol>
</div>
</li>
<li>
<p><code>primitive</code> primitive types (number, string etc.) Special features:</p>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
indexed
</td>
<td class="hdlist2">
<p>defined what type is returned in case of index access</p>
</td>
</tr>
<tr>
<td class="hdlist1">
autoboxedType
</td>
<td class="hdlist2">
<p>defines to which type the primitive can be auto boxed</p>
</td>
</tr>
<tr>
<td class="hdlist1">
assignmnentCompatible
</td>
<td class="hdlist2">
<p>defines to which type the primitive is assignment compatible</p>
</td>
</tr>
</table>
</div>
</li>
<li>
<p>types <code>any</code>, <code>null</code>, <code>void</code>, <code>undefined</code> – special types.</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>Annotations are not supported in the types DSL.</p>
</div>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Type_System_Implementation"><a class="anchor" href="#sec:Type_System_Implementation"></a><a class="link" href="#sec:Type_System_Implementation">6.2. Type System Implementation</a></h3>
<div class="paragraph">
<p>The bulk of the type system&#8217;s functionality is implemented in packages <code>org.eclipse.n4js.typesystem[.constraints|.utils]</code>.
Client code, e.g. in validations, should only access the type system through the facade class <code>N4JSTypeSystem</code>.
Each of the main type system functions, called "judgments", are implemented in one of the concrete subclasses of
base class <code>AbstractJudgment</code>. Internally, the type system is using a constraint solver for various purposes;
entry point for this functionality is class <code>InferenceContext</code>. All these classes are a good entry point into
the code base, for investigating further details.</p>
</div>
<div class="paragraph">
<p>Some type information is cached (e.g., the type of an expression in the AST) and the above facade will take care
to read from the cache instead of re-computing the information every time, as far as possible. This type cache is
being filled in a dedicated phase during loading and processing of an N4JS resource;
see <a href="#sec:Type_Inference_combined_with_AST_Traversal">Type Inference of AST</a> and <a href="#sec:N4JS_Resource_Load_States">N4JS Resource Load States</a> for details.</p>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Type_Inference_combined_with_AST_Traversal"><a class="anchor" href="#sec:Type_Inference_combined_with_AST_Traversal"></a><a class="link" href="#sec:Type_Inference_combined_with_AST_Traversal">6.3. Type Inference of AST</a></h3>
<div class="paragraph">
<p>Most judgments provided by the facade <code>N4JSTypeSystem</code> and implemented by subclasses of <code>AbstractJudgment</code> are used
ad-hoc whenever client code requires the information they provide. This is applied, in particular, to judgments</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>subtype</code></p>
</li>
<li>
<p><code>substTypeVariables</code></p>
</li>
<li>
<p><code>upperBound</code> / <code>lowerBound</code></p>
</li>
</ul>
</div>
<div class="paragraph">
<p>For judgment <code>type</code> footnote:<code class="Currently only for [language-n4js">type</code>, not for <code class="language-n4js">expectedType</code>,
but this may be changed in a future, further refactoring.], however, the processing is very different: we make
sure that the entire AST, i.e. all typable nodes of the AST, will be typed up-front in a single step, which
takes place during the <em>post-processing step</em> of an N4JSResource (see <a href="#sec:N4JS_Resource_Load_States">N4JS Resource Load States</a>), which
also has a couple other responsibilities. By triggering post-processing when client code invokes the type judgment
for the first time for some random AST node (at the latest; usually it is triggered earlier), we make sure that
this sequence is always followed.</p>
</div>
<div class="paragraph">
<p>The remainder of this section will explain this single-step typing of the entire AST in detail.</p>
</div>
<div class="sect3">
<h4 id="sec:Type_Inference_combined_with_AST_Traversal__Background"><a class="anchor" href="#sec:Type_Inference_combined_with_AST_Traversal__Background"></a><a class="link" href="#sec:Type_Inference_combined_with_AST_Traversal__Background">6.3.1. Background</a></h4>
<div class="paragraph">
<p>Originally, the N4JS type system could be called with any <code>EObject</code> at any given time, without any knowledge of the
context. While this looked flexible in the beginning, it caused severe problems solving some inference cases, e.g.,
the rule environment had to be prepared from the outside and recursion problems could occur, and it was also
inefficient because some things had to be recalculated over and over again (although caching helped).</p>
</div>
<div class="paragraph">
<p>It is better to do type inferencing (that is, computing the type of expressions in general) in a controlled manner.
That is, instead of randomly computing the type of an expression in the AST, it is better to traverse the AST in a
well-defined traversal order. That way, it is guaranteed that certain other nodes have been visited and, if not,
either some special handling can kick in or an error can be reported. This could even work with XSemantics and the
declarative style of the rules. The difference is that by traversing the AST in a controlled manner, the rules can
make certain assumptions about the content of the rule-environment, such as that it always contains information
about type variable bindings and that it always contains information about expected types etc.</p>
</div>
<div class="paragraph">
<p>In that scenario, all AST nodes are visited and all types (and expected types) are calculated up-front. Validation
and other parts then do not need to actually compute types (by calling the actual, Xsemantics-generated type system);
instead, at that time all types have already been calculated and can simply be retrieved from the cache (this is
taken care of by the type system facade <code>N4JSTypeSystem</code>).</p>
</div>
<div class="paragraph">
<p>This also affects scoping, since all cross-references have to be resolved in this type computation step. However,
even for scoping this has positive effects: E.g., the receiver type in property access expressions is always visited
<em>before</em> visiting the selector. Thus it is not necessary to re-calculate the receiver type in order to perform scoping
for the selector.</p>
</div>
<div class="paragraph">
<p>The above refactoring was done in summer 2015. After this refactoring, we are still using Xsemantics to compute the
types, i.e. the <code>type</code> judgement in Xsemantics was largely kept as before. However, the type judgment is invoked
in a controlled traversal order for each typable AST node in largely one step (controlled by <code>ASTProcessor</code> and <code>TypeProcessor</code>).</p>
</div>
<div class="paragraph">
<p>The upshot of this one-step type inference is that once it is completed, the type for every typable AST node is known.
Instead of storing this information in a separate model, this information will be stored and persisted in the type model
directly, as well as in transient fields of the AST <sup class="footnote">[<a id="_footnoteref_1" class="footnote" href="#_footnote_1" title="View footnote.">1</a>]</sup>. Currently, this applies only to types, not expected types;
the inference of expected types could / should be integrated into the one-step inference as part of a future, further
refactoring.</p>
</div>
<table id="tab:typeInferenceBeforeAfter" class="tableblock frame-all grid-all spread">
<caption class="title">Table 2. Comparison of inference of type of AST nodes before / after refactoring.</caption>
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Before</th>
<th class="tableblock halign-left valign-top">After</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">ad-hoc type inference (when client code needs the type information)</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">up-front type inference (once for entire AST;
later only reading from cache)</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">started anywhere</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">starts with root, i.e. the <code>Script</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Xsemantics rules traverse the AST at will, uncontrolled</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">well-defined, controlled traversal order</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">lazy, on-demand resolution of <code>ComputedTypeRef</code>s (they contain the resolution logic)</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">pro-active resolution of
<code>DeferredTypeRef</code>s (they themselves are dumb)</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect3">
<h4 id="sec:Triggering_Type_Inference_of_AST"><a class="anchor" href="#sec:Triggering_Type_Inference_of_AST"></a><a class="link" href="#sec:Triggering_Type_Inference_of_AST">6.3.2. Triggering</a></h4>
<div class="paragraph">
<p>The up-front type inference of the entire AST is part of the post-processing of every N4JSResource and is thus
triggered when post-processing is triggered. This happens when</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>someone directly calls <code>#performPostProcessing()</code> on an N4JSResource</p>
</li>
<li>
<p>someone directly calls <code>#resolveAllLazyCrossReferences()</code> on an N4JSResource,</p>
</li>
<li>
<p>EMF automatically resolves the first proxy, i.e. someone calls an EMF-generated getter for a value that is a proxy,</p>
</li>
<li>
<p>someone asks for a type for the first time, i.e. calls <code>N4JSTypeSystem#type()</code>,</p>
</li>
<li>
<p>&#8230;&#8203;</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>Usually this happens after the types builder was run with <code>preLinking==false</code> and before validation takes place.
For details, see classes <code>PostProcessingAwareResource</code> and <code>N4JSPostProcessor</code>.</p>
</div>
</div>
<div class="sect3">
<h4 id="sec:Traversal_Order_During_Type_Inference_of_AST"><a class="anchor" href="#sec:Traversal_Order_During_Type_Inference_of_AST"></a><a class="link" href="#sec:Traversal_Order_During_Type_Inference_of_AST">6.3.3. Traversal Order</a></h4>
<div class="paragraph">
<p>The traversal order during post-processing is a bit tricky, as some things need to be done in a top-down order (only
few cases, for now <sup class="footnote">[<a id="_footnoteref_2" class="footnote" href="#_footnote_2" title="View footnote.">2</a>]</sup>), others in a bottom-up order (e.g. the main typing of AST nodes),
and there is a third case in which several AST nodes are processed together (constraint-based type inference).</p>
</div>
<div class="paragraph">
<p>Figure <a href="#fig:traversalOrder">Order in which AST nodes are being processed during post-processing.</a> provides an example of an AST and shows in which order the nodes are processed. Green
numbers represent top-down processing, red numbers represent bottom-up processing and blue numbers represent the
processing of the surrounding yellow nodes in a single step.</p>
</div>
<div id="fig:traversalOrder" class="imageblock">
<div class="content">
<img src="chapters/06_typesystem/images/traversalOrder.png" alt="traversalOrder">
</div>
<div class="title">Figure 9. Order in which AST nodes are being processed during post-processing.</div>
</div>
<div class="paragraph">
<p>In the code, this is controlled by class <code>ASTProcessor</code>. The two main processing methods are</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>#processNode_preChildren()</code>, which will be invoked for all AST nodes in a top-down order (so top-down processing should be put here),</p>
</li>
<li>
<p><code>#processNode_postChildren()</code>, which will be invoked for all AST nodes in a bottom-up order (so bottom-up processing should be put here).</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>The common processing of groups of adjacent yellow nodes (represented in the figure by the two yellow/brown
triangles) is achieved by <code>PolyProcessor</code> telling the <code>TypeProcessor</code> to</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>ignore certain nodes (all yellow nodes) and</p>
</li>
<li>
<p>invoke method <code>PolyProcessor#inferType()</code> for the root yellow node in each group (only the root!).
.
For details, see the two methods <code>#isResponsibleFor()</code> and <code>#isEntryPoint()</code> in <code>PolyProcessor</code>.</p>
</li>
</ol>
</div>
</div>
<div class="sect3">
<h4 id="sec:Cross_References_During_Type_Inference_of_AST"><a class="anchor" href="#sec:Cross_References_During_Type_Inference_of_AST"></a><a class="link" href="#sec:Cross_References_During_Type_Inference_of_AST">6.3.4. Cross-References</a></h4>
<div class="paragraph">
<p>While typing the entire AST, cross-references need special care. Three cases of cross-references need to be distinguished:</p>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
backward reference
</td>
<td class="hdlist2">
<p>= cross-reference within the same file to an AST node that was already processed</p>
<div class="ulist">
<ul>
<li>
<p>always legal</p>
</li>
<li>
<p>processing: simply read the type from the cache that is currently being filled</p>
</li>
</ul>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
forward reference
</td>
<td class="hdlist2">
<p>= cross-reference within the same file to an AST node that was not yet processed</p>
<div class="ulist">
<ul>
<li>
<p>usually illegal<br>
exception: legal if reference points to an <em>identifiable subtree</em> (a subtree of an AST with an identifiable element at its root)</p>
</li>
<li>
<p>processing: forward process the identifiable subtree and report back the type of its root</p>
</li>
</ul>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
references to other files
</td>
<td class="hdlist2">
<div class="ulist">
<ul>
<li>
<p>always legal, because they can, by nature, only point to an identifiable subtree in the other file&#8217;s AST</p>
</li>
<li>
<p>processing: either &#8230;&#8203;</p>
<div class="ulist">
<ul>
<li>
<p>read type from TModule obtained from index (if available), or</p>
</li>
<li>
<p>load other file from source, trigger its post-processing (if not in progress or completed already), forward process the identifiable subtree (if not processed already) and report back the type of its root.</p>
</li>
</ul>
</div>
</li>
</ul>
</div>
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>Note that for references to an ancestor (upward references) or successor (downward references) within an AST, the
classification as a forward or backward reference depends on whether we are in top-down or bottom-up processing.
Figure <a href="#fig:upwardDownward">Upward Downward</a> illustrates this: the left and right side show the same AST but on the
left side we assume a top down processing whereas on the right we assume a bottom up processing. On both sides,
backward references are shown in green ink (because they are unproblematic and always legal) and forward references
are shown in red ink. Now, looking at the two arrows pointing from a node to its parent, we see that it is classified
as a backward reference on the left side (i.e. top down case) but as a forward reference on the right side (i.e. bottom
down case). Conversely, an arrow from a node to its child is classified as a forward reference on the left side and as
a backward reference on the right side. Arrows across subtrees, however, are classified in the same way on the left and
right side (see the horizontal arrows at the bottom).</p>
</div>
<div id="fig:upwardDownward" class="imageblock">
<div class="content">
<img src="chapters/06_typesystem/images/upwardDownward.png" alt="upwardDownward">
</div>
<div class="title">Figure 10. Backward and forward references in top-down and bottom-up processing.</div>
</div>
</div>
<div class="sect3">
<h4 id="sec:Function_Accessor_Bodies_During_Type_Inference_of_AST"><a class="anchor" href="#sec:Function_Accessor_Bodies_During_Type_Inference_of_AST"></a><a class="link" href="#sec:Function_Accessor_Bodies_During_Type_Inference_of_AST">6.3.5. Function/Accessor Bodies</a></h4>
<div class="paragraph">
<p>An important exception to the basic traversal order shown in Figure <a href="#fig:traversalOrder">Order in which AST nodes are being processed during post-processing.</a> is that the body of all
functions (including methods) and field accessors is postponed until the end of processing. This is used to avoid
unnecessary cycles during type inference due to a function&#8217;s body making use of the function itself or some other
declarations on the same level as the containing function. For example, the following code relies on this:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-n4js" data-lang="n4js">let x = f();
function f(): X {
if(x) {
// XPECT noerrors --&gt; "any is not a subtype of X." at "x"
return x;
}
return new X();
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>Similar situation using fields and methods:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-n4js" data-lang="n4js">class C {
d = new D();
mc() {
// XPECT noerrors --&gt; "any is not a subtype of D." at "this.d"
let tmp: D = this.d;
}
}
class D {
md() {
new C().mc();
}
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>For details of this special handling of function bodies, see method <code>ASTProcessor#isPostponedNode(EObject)</code> and field
<code>ASTMetaInfoCache#postponedSubTrees</code> and the code using it. For further investigation, change <code>isPostponedNode()</code> to always
return false and debug with the two examples above (which will then show the incorrect errors mentioned in the XPECT
comments) or run tests to find more cases that require this handling.</p>
</div>
</div>
<div class="sect3">
<h4 id="sec:Poly_Expressions_During_Type_Inference_of_AST"><a class="anchor" href="#sec:Poly_Expressions_During_Type_Inference_of_AST"></a><a class="link" href="#sec:Poly_Expressions_During_Type_Inference_of_AST">6.3.6. Poly Expressions</a></h4>
<div class="paragraph">
<p>Polymorphic expressions, or <em>poly expressions</em> for short, are expressions for which the actual type depends on the
expected type and/or the expected type depends on the actual type. They require constraint-based type inference
because the dependency between the actual and expected type can introduce dependency cycles between the types of
several AST nodes which are best broken up by using a constraint-based approach. This is particularly true when
several poly expressions are nested. Therefore, poly expressions are inferred neither in top-down nor in bottom-up
order, but all together by solving a single constraint system.</p>
</div>
<div class="paragraph">
<p>Only a few types of expressions can be polymorphic; they are called <em>poly candidates</em>: array literals, object literals,
call expressions, and function expressions. The following rules tell whether a poly candidate is actually poly:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>ArrayLiteral</code> — always poly (because their type cannot be declared explicitly).</p>
</li>
<li>
<p><code>ObjectLiteral</code> — if one or more properties do not have a declared type.</p>
</li>
<li>
<p><code>CallExpression</code> — if generic &amp; not parameterized.</p>
</li>
<li>
<p><code>FunctionExpression</code> — if return type or type of one or more formal parameters is undeclared.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>This is a simplified overview of these rules, for details see method <code>#isPoly(Expression)</code> in <code>AbstractPolyProcessor</code>.</p>
</div>
<div class="paragraph">
<p>The main logic for inferring the type of poly expressions is found in method <code>#inferType()</code> in class <code>PolyProcessor</code>.
It is important to note that this method will only be called for root poly expressions (see above). In short, the basic
approach is to create a new, empty <code>InferenceContext</code>, i.e. constraint system, add inference variables and constraints for
the root poly expression and all its nested poly expressions, solve the constraint system and use the types in the solution
as the types of the root and nested poly expressions. For more details see method <code>#inferType()</code> in class <code>PolyProcessor</code>.</p>
</div>
<div class="paragraph">
<p>So, this means that nested poly expressions do not introduce a new constraint system but instead simply extend their parent
poly’s constraint system by adding additional inference variables and constraints. <strong>But not every nested expression that is
poly is a nested poly expression in that sense!</strong> Sometimes, a new constraint system has to be introduced. For example:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>child poly expressions that appear as argument to a call expression are nested poly expressions (i.e. inferred in same constraint system as the parent call expression),</p>
</li>
<li>
<p>child poly expressions that appear as target of a call expression are <strong>not</strong> nested poly expressions and a new constraint system has to be introduced for them.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>For details see method <code>#isRootPoly()</code> in <code>AbstractPolyProcessor</code> and its clients.</p>
</div>
</div>
<div class="sect3">
<h4 id="sec:Constraint_Solver_used_During_Type_Inference_of_AST"><a class="anchor" href="#sec:Constraint_Solver_used_During_Type_Inference_of_AST"></a><a class="link" href="#sec:Constraint_Solver_used_During_Type_Inference_of_AST">6.3.7. Constraint Solver</a></h4>
<div class="paragraph">
<p>The simple constraint solver used by the N4JS type system, mainly for the inference of poly expressions, is implemented
by class <code>InferenceContext</code> and the other classes in package <code>org.eclipse.n4js.typesystem.constraints</code>.</p>
</div>
<div class="paragraph">
<p>The constraint solving algorithm used here is largely modeled after the one defined in <code>The Java Language Specification 8</code>,
Chapter 18, but was adjusted in a number of ways, esp. by removing functionality not required for N4JS (e.g. primitive types,
method overloading) and adding support for specific N4JS language features (e.g. union types, structural typing).</p>
</div>
<div class="paragraph">
<p>For details see the API documentation of class <code>InferenceContext</code>.</p>
</div>
</div>
<div class="sect3">
<h4 id="sec:Type_Guards_During_Type_Inference_of_AST"><a class="anchor" href="#sec:Type_Guards_During_Type_Inference_of_AST"></a><a class="link" href="#sec:Type_Guards_During_Type_Inference_of_AST">6.3.8. Type Guards</a></h4>
<div class="paragraph">
<p>During AST post-processing, the control and data flow analyses are performed.
This means, that first a flow graph is created and then traversed.
During the traversal, a type guard analysis is performed which saves information by evaluating <code>instanceof</code> expressions.
As a result, the analysis provides a reliable set of RHS expressions of <code>instanceof</code> expressions for each AST element of type <code>IdentifierRef</code>.</p>
</div>
<div class="paragraph">
<p>This set is evaluated in the <code>TypeJudgments.java</code> when typing <code>IdentifierRef</code> elements.
In case the set is not empty, the types of all elements is calculated.
The type of the <code>IdentifierRef</code> will then become the intersection of its original type and all types previously calculated.</p>
</div>
</div>
</div>
<div class="sect2">
<h3 id="sec:Structural_Typing"><a class="anchor" href="#sec:Structural_Typing"></a><a class="link" href="#sec:Structural_Typing">6.4. Structural Typing</a></h3>
<div class="paragraph">
<p>Structural typing as an optional subtyping mode in N4JS is implemented in <code>StructuralTypingComputer</code>, activated depending on
the value of property <code>typingStrategy</code> in <code>ParameterizedTypeRef</code> and its subclasses.</p>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_type-index"><a class="anchor" href="#_type-index"></a><a class="link" href="#_type-index">7. Type Index</a></h2>
<div class="sectionbody">
<div class="sect2 language-n4js">
<h3 id="sec:Type_Index_Design_Rationale"><a class="anchor" href="#sec:Type_Index_Design_Rationale"></a><a class="link" href="#sec:Type_Index_Design_Rationale">7.1. Design Rationale</a></h3>
<div class="paragraph">
<p>We use a separate types model to represent types, see <a href="#sec:Type_Model_and_Grammar">Type Model and Grammar</a>. Declared elements (e.g., classes)
in N4JS are parsed and a new types model instance is derived from them. All type references (of the N4JS <a href="#AC">AST</a>)
are then bound to these type instances and not to the N4JS declaration. However, there exists a relation between a type
and its declaration. The type instances (which are EObjects) are added to the resource of the N4JS file as part of
the public interface of the resource. This public interface is represented by a <code>TModule</code>. While the actual source code
is the first element of a resource (index 0), the module is stored at index 1. It contains the derived type information,
the information about exported variables and functions as well as information about the project and vendor. The Xtext
serializer ignores the additional element. Besides, the complete type instances are stored in the user data section of
the <code>IEObjectDescription</code> of the <code>TModule</code>. Since the user data only allows strings to be stored, the EObjects are serialized
(within a virtual resource). When a reference is then bound to a type, the type can be directly recreated (deserialized)
from the user data. The deserialized EObject is then added to the appropriate resource. It is not necessary to load the
complete file just to refer to a type from that file.</p>
</div>
<div class="paragraph">
<p>The design relies on two key features of Xtext:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Besides the parsed text (i.e., the AST), other elements can be stored in a resource, which are then ignored by
the Xtext serializer, while still being properly contained in an EMF resource.</p>
</li>
<li>
<p>The <code>DerivedStateAwareResource</code> allows some kind of post processing steps when reading a resource. This enables a custom
class, here <code>N4JSDerivedStateComputer</code>, to create the types models (TClass, TRole and so on) from the parsed <code>N4ClassDeclaration</code>,
<code>N4RoleDeclaration</code> and so on.</p>
</li>
</ol>
</div>
<div class="sect3">
<h4 id="sec:Getting_the_Xtext_Index_Content_IResourceDescriptions"><a class="anchor" href="#sec:Getting_the_Xtext_Index_Content_IResourceDescriptions"></a><a class="link" href="#sec:Getting_the_Xtext_Index_Content_IResourceDescriptions">7.1.1. Getting the Xtext Index (<code>IResourceDescriptions</code>) Content</a></h4>
<div class="paragraph">
<p>An instance of the <code>IResourceDescriptions</code> can be acquired from the resource description provider. Just like all services
in Xtext, this can be injected into the client code as well. The resource descriptions accepts a non-null resource set.
The resource set argument is mandatory to provide the index with the proper state. We are differentiating between three
different state. The first one is the persisted one, basically the builder state is a resource description as well, and
it provides a content that is based on the persisted state of the files (in our case the modules and package.json file)
on the file system. The second one is the live scoped index, this is modification and dirty state aware. Namely when using
this resource descriptions then an object description will be searched in the resource set itself first, then in the dirty
editor’s index, finally among the persisted ones. The third index is the named builder scoped one. This index should not be
used by common client code, since it is designed and implemented for the Xtex builder itself.</p>
</div>
<div class="paragraph">
<p>A resource set and hence the index can be acquired from the N4JS core, in such cases an optional N4JS project can be specified.
The N4JS project argument is used to retrieve the underlying Eclipse resource project (if present) and get the resource set from
the resource set provider. This is completely ignored when the application is running in headless mode and Eclipse resource
projects are not available. It is also important to note that the resource set is always configured to load only the persisted
states.</p>
</div>
<div class="paragraph">
<p>When the Eclipse platform is running, the workspace is available and the all N4JS projects are backed by an Eclipse resource
project. With the Eclipse resource project the resource sets can be initialized properly via the resource set initializer
implementations. This mechanism is used to get the global objects (such as console) and the built-in types (such as string,
number) into the resource set via the corresponding resource set adapters. In the headless case a special resource set
implementation is used; <code>ResourceSetWithBuiltInScheme</code>. This implementation is responsible to initialize the globals and the
built-in types into itself.</p>
</div>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Design_Overview"><a class="anchor" href="#sec:Design_Overview"></a><a class="link" href="#sec:Design_Overview">7.2. Design Overview</a></h3>
<div class="paragraph">
<p><a href="#fig:cd_TypeModelWithXtextIndex">Type Model With Xtext Index</a> shows a simplified UML class diagram with the involved
classes. In the figure, a class (defined as N4ClassExpression in the <a href="#AC">AST</a> and its type TClass) is used as a sample,
declared type—roles or enums are handled similarly.</p>
</div>
<div class="paragraph">
<p>In the Eclipse project build the <code>N4JSResourceDescriptionManager</code> (resp. by the logic of its super class) is called by the
<code>N4JSGenerateImmediatelyBuilderState</code> to get the resource description for a resource. The resource description manager loads
the resource to create / update the resource descriptions. Loading an Xtext resource means that it is reparsed again.
All cross references are handled here only by the lazy linker so that the node model will contain an unresolved proxy
for all cross references.</p>
</div>
<div class="paragraph">
<p>After the resource is loaded there is a derived state installed to the resource. For this the <code>N4JSDerivedStateComputer</code> will
be called. It will take the parse result (= EObject tree in first slot of the resource) and navigate through these objects
to create type trees for each encountered exportable object that are stored in exported <code>TModule</code> of the resource.
<a href="#fig:cd_CreateTypeFromAST">Create Type From AST</a>, a snippet of <a href="#fig:cd_TypeModelWithXtextIndex">Type Model with Xtext Index</a>,
shows only the classes involved when creating the types from the resource.</p>
</div>
<div id="fig:cd_CreateTypeFromAST" class="imageblock center">
<div class="content">
<img src="chapters/07_typeindex/images/cd_CreateTypeFromAST.png" alt="cd CreateTypeFromAST">
</div>
<div class="title">Figure 11. Type From AST</div>
</div>
<div id="fig:cd_TypeModelWithXtextIndex" class="imageblock center">
<div class="content">
<img src="chapters/07_typeindex/images/cd_CreateTypeFromIndex.png" alt="cd CreateTypeFromIndex">
</div>
<div class="title">Figure 12. Create Type From Index</div>
</div>
<div id="fig:cd_SerializeToIndex" class="imageblock center">
<div class="content">
<img src="chapters/07_typeindex/images/cd_SerializeToIndex.png" alt="cd SerializeToIndex">
</div>
<div class="title">Figure 13. Serialize To Index</div>
</div>
<div class="paragraph language-javascript">
<p>For these elements types have to be derived as they are exportable: <code>N4ClassDeclaration</code>, <code>N4RoleDeclaration</code>, <code>N4InterfaceDeclaration</code>,
<code>N4EnumDeclaration</code>, <code>ExportedVariableDeclaration</code> and <code>FunctionDeclaration</code>.</p>
</div>
<div class="paragraph">
<p>After loading and initializing the resources now all cross references in the resources are resolved. For this the
<code>ErrorAwareLinkingService</code> is used. This class will in turn call the <code>N4JSScopeProvider</code> to first try to do scoping locally
but eventually also delegate to the global scope provider to find linked elements outside the current resource. This
will be done e.g. for every import statement inside the N4JS resource.</p>
</div>
<div class="paragraph">
<p>For determine the global scope all visible containers for this resource are calculated. For this the project description
(= loaded package.json file) is used to determine which folders of the current project should be included for looking for
N4JS resources. Also all referenced projects and their resources are added to the visible containers. For these containers
<code>N4JSGlobalScopeProvider</code> builds up a container scope. This container scope will be a <code>N4JSTypesScope</code> instance.</p>
</div>
<div class="paragraph">
<p>For the actual linked element in the resource to be resolved, its fully qualified name is used. This name is calculated by
using the <code>IQualifiedNameConverter</code>. We bound a custom class named <code>N4JSQualifiedNameConverter</code> who converts the <code>/</code> inside the
qualified name to a dot, so e.g. <code>my/module/MyFileName</code> is converted to <code>my.module.MyFileName</code>. Btw. the initial qualified name
was derived from the node model.</p>
</div>
<div class="paragraph">
<p>With this qualified name <code>N4JSTypeScope.getSingleElement</code> is called. This method does the actual resolving of the cross reference.
For this the URI of the cross reference is used to determine the linked resource.</p>
</div>
<div class="paragraph">
<p>There are now three cases:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>If the resource which contains the linked EObject is already loaded the EObject description found for the URI is returned</p>
</li>
<li>
<p>If the resource is not loaded but the first slot of the resource is empty the referenced type is tried to be rebuild from
an existing resource description for the linked resource inside the Xtext index.</p>
</li>
<li>
<p>If the resource is not loaded and the first slot is set, the linked EObject will be resolved with the fragment of the
given URI.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>While calculating the resource description for a <code>N4JSResource</code>, the EObject descriptions of their exported objects have to be
calculated as well. For this the <code>N4JSResourceDescriptionStrategy</code> is used. For computing the exported objects of a resource only
the root <code>TModule</code> and its contained types and variables are taken in consideration.</p>
</div>
<div class="paragraph">
<p>The EObjectDescriptions for a n4js resource include:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>An exported representation of the derived <code>TModule</code>. This carries these properties:</p>
<div class="ulist">
<ul>
<li>
<p>a qualified name (e.g. <code>my.module.MyFileName</code> when the resource is stored under <code>src/my/module/MyFileName.js</code> in the project and
the project description has marked src has src folder). The calculation of the qualified name is delegated to the <code>N4JSNamingUtil</code>.</p>
</li>
<li>
<p>the user data which is the serialized string of the exported <code>TModule</code> itself. It includes the types determined for this
resource, so for every element found in this resource, that is contained in an <code>ExportStatement</code>, an EObject has been created
before in <code>N4JSDerivedStateComputer</code>. In most cases this an EObject extending <code>Type</code> from the types model, e.g. <code>TClass</code> for
<code>N4ClassDeclaration</code>. There is an exception for <code>ExportedVariableDeclaration</code> where <code>TVariable</code> is used as representative (and this
EObject is not contained in the types model only in the N4JS model). For usability reasons (quicker quick fix etc.), also
top level types not exported yet are stored in the <code>TModel</code>.</p>
</li>
<li>
<p>the information on project and vendor id are part of the module descriptor.</p>
</li>
</ul>
</div>
</li>
<li>
<p>Descriptions for all top level types that are defined in the resource. These descriptions do not have any special properties,
so they just have a name.</p>
</li>
<li>
<p>All exported variables are also described in the resource description. They don’t carry any special information either.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>The EObjectDescription for an EObject contained in an <code>ExportStatement</code>:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>the qualified name of the module export (e.g. for a <code>N4ClassDeclaration</code> the qualified name <code>my.module.MyFileName.MyClassName</code> would
be produced, when the resource is stored under <code>src/my/module/MyFileName.js</code> in the project, the project description has marked
src has src folder and the N4 class uses the name MyClassName]). The calculation of the qualified name is delegated to the
<code>N4JSNamingUtil</code>.</p>
</li>
<li>
<p>the EObject represented by the EObject description, here this is not the actual EObject from N4JS but the type EObject from
the TypeSystem, that has been inferenced by using <code>N4JSTypeInferencer</code></p>
</li>
<li>
<p>the user data is only an empty map for this EObjectDescription</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>With this the resource description for a resource should be fully created / updated. <a href="#fig:cd_SerializeToIndex">Serialize to Index</a>
shows the classes involved creating the resource and EObjectDescriptions, along with the serialized type information.</p>
</div>
</div>
<div class="sect2">
<h3 id="sec:N4JS_Resource_Load_States"><a class="anchor" href="#sec:N4JS_Resource_Load_States"></a><a class="link" href="#sec:N4JS_Resource_Load_States">7.3. N4JS Resource Load States</a></h3>
<div class="paragraph">
<p>Below state diagram depicts the state transitions when loading and resolving an N4JS resource.</p>
</div>
<div class="imageblock center">
<div class="content">
<img src="chapters/07_typeindex/images/resource_set_load_states.png" alt="resource set load states">
</div>
<div class="title">Figure 14. N4JS Resource resolution states</div>
</div>
<div class="paragraph">
<p>Additionally, the following table relates the values of the resource&#8217;s flags to the states.</p>
</div>
<table class="tableblock frame-all grid-all spread">
<colgroup>
<col style="width: 11.1111%;">
<col style="width: 11.1111%;">
<col style="width: 11.1111%;">
<col style="width: 11.1111%;">
<col style="width: 11.1111%;">
<col style="width: 11.1111%;">
<col style="width: 11.1111%;">
<col style="width: 11.1111%;">
<col style="width: 11.1112%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">State</th>
<th class="tableblock halign-left valign-top">Parse Result</th>
<th class="tableblock halign-left valign-top">AST</th>
<th class="tableblock halign-left valign-top">TModule</th>
<th class="tableblock halign-left valign-top">ASTMetaInfoCache</th>
<th class="tableblock halign-left valign-top">loaded</th>
<th class="tableblock halign-left valign-top">fullyInitialized</th>
<th class="tableblock halign-left valign-top">fullyProcessed</th>
<th class="tableblock halign-left valign-top">reconciled</th>
</tr>
</thead>
<tbody>
<tr>
<th class="tableblock halign-left valign-top"><p class="tableblock">Created</p></th>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>null</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>null</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>null</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>null</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">false</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">false</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">false</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">false</p></td>
</tr>
<tr>
<th class="tableblock halign-left valign-top"><p class="tableblock">Created'</p></th>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>null</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>null</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>null</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>null</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">false</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">true</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">false</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">false</p></td>
</tr>
<tr>
<th class="tableblock halign-left valign-top"><p class="tableblock">Loaded</p></th>
<td class="tableblock halign-left valign-top"><p class="tableblock">available</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">with lazy linking proxies</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>null</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>null</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">true</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">false</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">false</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">false</p></td>
</tr>
<tr>
<th class="tableblock halign-left valign-top"><p class="tableblock">Pre-linked</p></th>
<td class="tableblock halign-left valign-top"><p class="tableblock">available</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">with lazy linking proxies</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">with stubs</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>null</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">true</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">true</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">false</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">false</p></td>
</tr>
<tr>
<th class="tableblock halign-left valign-top"><p class="tableblock">Fully Initialized</p></th>
<td class="tableblock halign-left valign-top"><p class="tableblock">available</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">with lazy linking proxies</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">with DeferredTypeRefs</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>null</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">true</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">true</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">false</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">false</p></td>
</tr>
<tr>
<th class="tableblock halign-left valign-top"><p class="tableblock">Fully Processed</p></th>
<td class="tableblock halign-left valign-top"><p class="tableblock">available</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">available</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">available</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">available</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">true</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">true</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">true</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">false</p></td>
</tr>
<tr>
<th class="tableblock halign-left valign-top"><p class="tableblock">Loaded from Description</p></th>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>null</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">proxy</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">available</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>null</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">indeterminate</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">true</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">true</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">false</p></td>
</tr>
<tr>
<th class="tableblock halign-left valign-top"><p class="tableblock">Loaded from Description'</p></th>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>null</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">proxy</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">with DeferredTypeRefs</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>null</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">indeterminate</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">true</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">true</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">false</p></td>
</tr>
<tr>
<th class="tableblock halign-left valign-top"><p class="tableblock">Fully Initialized ®</p></th>
<td class="tableblock halign-left valign-top"><p class="tableblock">available</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">with lazy linking proxies</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">available</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>null</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">indeterminate</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">true</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">false</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">true</p></td>
</tr>
<tr>
<th class="tableblock halign-left valign-top"><p class="tableblock">Fully Processed ®</p></th>
<td class="tableblock halign-left valign-top"><p class="tableblock">available</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">available</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">available</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">available</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">indeterminate</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">true</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">true</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">true</p></td>
</tr>
</tbody>
</table>
<div class="paragraph">
<p>Remarks:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>oddities are shown in red ink, in the above figure.</p>
</li>
<li>
<p>in the above figure:</p>
<div class="ulist">
<ul>
<li>
<p>"AST (proxy)" means the AST consists of only a single node of type <code>Script</code> and that is a proxy,</p>
</li>
<li>
<p>"AST (lazy)" means the AST is completely created, but cross-references are represented by unresolved
Xtext lazy-linking proxies,</p>
</li>
<li>
<p>"TModule (stubs)" means the TModule has been created with incomplete information, e.g. return types of
all TMethods/TFunctions will be <code>null</code> (only used internally by the incremental builder),</p>
</li>
<li>
<p>"TModule (some deferred)" means the TModule has been created, does not contain stub, but some
`TypeRef`s are `DeferredTypeRef`s that are supposed to be replaced by proper `TypeRef`s during post-processing.</p>
</li>
<li>
<p>"AST" and "TModule" means the AST/TModule is available without any qualifications.</p>
</li>
</ul>
</div>
</li>
<li>
<p>state <strong>Created'</strong>: only required because Xtext does not clear flag <code>fullyInitialized</code> upon unload; that is done lazily
when <code>#load()</code> is invoked at a later time.. Thus, we do not reach state <strong>Created</strong> when unloading from state
<strong>Fully Initialized</strong> but instead get to state <strong>Created'</strong>. To reach state <strong>Created</strong> from <strong>Fully Initialized</strong> we have to
explicitly invoke <code>#discardDerivedState()</code> before(!) unloading.</p>
</li>
<li>
<p>state <strong>Loaded from Description'</strong>: transition <code>#unloadAST()</code> from state <strong>Fully Initialized</strong> leaks a non-post-processed
TModule into state <strong>Loaded from Description</strong>, which is inconsistent with actually loading a TModule from the index,
because those are always fully processed. Hence, the addition of state <strong>Loaded from Description'</strong>.</p>
</li>
<li>
<p>states <strong>Fully Initialized ®</strong> and <strong>Fully Processed ®</strong>: these states are reached via reconciliation of a pre-existing
TModule with a newly loaded AST. These states differ in an unspecified way from their corresponding non-reconciled
states. For example, in state <strong>Fully Initialized ®</strong> the TModule does not contain any DeferredTypeRefs while, at the
same time, the TModule isn&#8217;t fully processed, because proxy resolution, typing, etc. have not taken place, yet.</p>
</li>
<li>
<p>TODO old text (clarify this; I could not reproduce this behavior): when <code>unloadAST</code> is called, <code>fullyInitialized</code> remains
unchanged. This is why the value of <code>fullyInitialized</code> should be indeterminate in row <strong>Loaded from Description</strong>; it
depends on the previous value if the state <strong>Loaded from Description</strong> was reached by calling <code>unloadAST</code>.</p>
</li>
</ul>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Type_Builder"><a class="anchor" href="#sec:Type_Builder"></a><a class="link" href="#sec:Type_Builder">7.4. Types Builder</a></h3>
<div class="paragraph">
<p>When a resource is loaded, it is parsed, linked, post-processed, validated and eventually compiled. For linking and validation
type information is needed, and as described above the type information is created automatically when loading a resource using
the types builder. <a href="#fig:ad_resourceLoading">Resource Loading</a> shows an activity model with the different actions performed when
a resource is loaded.</p>
</div>
<div id="fig:ad_resourceLoading" class="imageblock center">
<div class="content">
<img src="chapters/07_typeindex/images/ad_resourceLoading.png" alt="ad resourceLoading">
</div>
<div class="title">Figure 15. Activity Diagram, Resource Loading</div>
</div>
<div class="paragraph">
<p>The blue colored steps are standard Xtext workflow. Handling the TModule and storing that in the index are N4 specific (red background).</p>
</div>
<div class="sect3">
<h4 id="sec:Type_Inference_not_allowed_in_Type_Builder"><a class="anchor" href="#sec:Type_Inference_not_allowed_in_Type_Builder"></a><a class="link" href="#sec:Type_Inference_not_allowed_in_Type_Builder">7.4.1. Type Inference not allowed in Types Builder</a></h4>
<div class="paragraph">
<p>A crucial point in the workflow described above is the combination of types model building and type inference. In some cases,
the type of a given element is not directly stated in the AST but has to be inferred from an expression and other types. For
example, when a variable declaration does not declare the variable’s type explicitly but provides an initializer expression,
the actual type of the variable is inferred to be the type of the expression.</p>
</div>
<div class="paragraph">
<p>However, the types builder cannot be allowed to use type inference, mainly for two reasons:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>type inference through Xsemantics could lead to resolution of cross-references (i.e. EMF proxies generated by the lazy
linker) and because the types builder is triggered when getContents() is called on the containing <code>N4JSResource</code> this would
break a basic contract of EMF resources.</p>
</li>
<li>
<p>type inference could cause other resources to be loaded which would lead to problems (infinite loops or strange results)
in case of circular dependencies. This is illustrated in <a href="#fig:sd_typesBuilder_problem">Types Builder Problem</a> and
<a href="#fig:sd_typesBuilder_proxies">Types Builder Proxies</a>.</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>Therefore, whenever the type of a particular element has to be inferred, the types builder will use a special type reference
called <code>DeferredTypeRef</code> <sup class="footnote">[<a id="_footnoteref_3" class="footnote" href="#_footnote_3" title="View footnote.">3</a>]</sup>,
in order to defer the actual type inference to a later stage, i.e. the post-processing stage.</p>
</div>
<div id="fig:sd_typesBuilder_problem" class="imageblock center">
<div class="content">
<img src="chapters/07_typeindex/images/sd_typesBuilder_problem.png" alt="sd typesBuilder problem">
</div>
<div class="title">Figure 16. Sequence Diagram, Types Builder Problem</div>
</div>
<div id="fig:sd_typesBuilder_proxies" class="imageblock center">
<div class="content">
<img src="chapters/07_typeindex/images/sd_typesBuilder_proxies.png" alt="sd typesBuilder proxies">
</div>
<div class="title">Figure 17. Sequence Diagram, Types Builder with Proxies</div>
</div>
</div>
<div class="sect3">
<h4 id="sec:ComputedTypeReferences"><a class="anchor" href="#sec:ComputedTypeReferences"></a><a class="link" href="#sec:ComputedTypeReferences">7.4.2. Deferred Type References</a></h4>
<div class="paragraph">
<p>Whenever type inference would be required to obtain the actual type of an element, the types builder will insert a stub to defer
actual type inference (see previous section). A dedicated subclass of <code>TypeRef</code>, called <code>DeferredTypeRef</code>, is used that contains neither
the actual type information nor any information necessary to perform the type inference at a later point in time. Later, this
<code>DeferredTypeRef</code> will be replaced during post-processing, see <code>TypeDeferredProcessor</code>.</p>
</div>
<div class="paragraph">
<p>All <code>DeferredTypeRef</code>s will be replaced by the actual types during post-processing. One important reason for resolving
all <code>DeferredTypeRef</code>s as early as possible is that they are not suited for serialization and therefore have to be removed
from the types model before populating the Xtext index, which includes serializing the TModule into the user data of the
root element. This is always assured by the logic that manages the triggering of the post-processing phase.</p>
</div>
<div class="paragraph">
<p>To manually trigger resolution of all <code>DeferredTypeRef</code>s in a given types model, simply call method <code>performPostProcessing(CancelIndicator)</code>
of the containing <code>N4JSResource</code> (should never be required by client code such as validations).</p>
</div>
</div>
<div class="sect3">
<h4 id="sec:Use_cases_of_ComputedTypeRef"><a class="anchor" href="#sec:Use_cases_of_ComputedTypeRef"></a><a class="link" href="#sec:Use_cases_of_ComputedTypeRef">7.4.3. Use cases of DeferredTypeRef</a></h4>
<div class="paragraph">
<p>Currently, <code>DeferredTypeRef</code>s are created by the types builder only in these cases:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>actual type of an exported TVariable if no declared type but an initialization expression are given.</p>
</li>
<li>
<p>actual type of a TField if no declared type but an initialization expression are given.</p>
</li>
<li>
<p>actual type of properties of ObjectLiterals if not declared explicitly.</p>
</li>
<li>
<p>actual type of formal parameters and return value of function expressions if not declared explicitly.</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>Note that this overview might easily get out-dated; see references to class <code>DeferredTypeRef</code> in the code.</p>
</div>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Incremental_Builder_Overview"><a class="anchor" href="#sec:Incremental_Builder_Overview"></a><a class="link" href="#sec:Incremental_Builder_Overview">7.5. Incremental Builder (Overview)</a></h3>
<div class="paragraph">
<p>This section provides a brief overview of how the incremental builder works.</p>
</div>
<div class="paragraph">
<p>General remarks:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>The N4JS incremental builder is a combination of Eclipse builder infrastructure, Xtext-specific builder functionality and
several adjustments for N4JS and N4MF.</p>
</li>
<li>
<p>The <code>IBuilderState</code> implementation is identical to the persisted Xtext index. No matter how many Xtext languages are supported
by the application, only a single <code>IBuilderState</code> instance is available in the application. Since we have one single <code>IBuilderState</code>,
we have one single persisted Xtext index throughout the application.</p>
</li>
<li>
<p>For simplicity, the below description assumes we have only N4JS projects in the workspace and no other Xtext languages are
installed.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Major components:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>XtextBuilder</code> (inherits from Eclipse’s <code>IncrementalProjectBuilder</code>):</p>
<div class="ulist">
<ul>
<li>
<p>the actual incremental builder</p>
</li>
<li>
<p>note: Eclipse will create one instance of <code>XtextBuilder</code> per project at startup.</p>
</li>
</ul>
</div>
</li>
<li>
<p><code>IBuilderState</code> (Xtext specific; no Eclipse pendant):<br>
identical to the <code>Xtext index</code>, i.e. the globally shared, persisted instance of<br>
<code>IResourceDescriptions</code>.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Main workflow:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>for each project that contains at least one resource that requires rebuilding, Eclipse will call the
project’s <code>XtextBuilder</code>.</p>
</li>
<li>
<p>each <code>XtextBuilder</code> will perform some preparations and will then delegate to <code>IBuilderState</code> which will iterate over
all resources in the builder’s project that require rebuilding.</p>
</li>
</ol>
</div>
<div class="sect3">
<h4 id="sec:Incremental_Builder_Overview__XtextBuilder"><a class="anchor" href="#sec:Incremental_Builder_Overview__XtextBuilder"></a><a class="link" href="#sec:Incremental_Builder_Overview__XtextBuilder">7.5.1. XtextBuilder</a></h4>
<div class="paragraph">
<p>Whenever a change in the workspace happens &#8230;&#8203;</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p><a id="itm:start"></a> Eclipse will collect all projects that contain changed resources and compute a project-level build
order (using the <code>build order</code> of the workspace, see <code>Workspace#getBuildOrder()</code>, which is based on project dependencies)</p>
</li>
<li>
<p>for the first <sup class="footnote">[<a id="_footnoteref_4" class="footnote" href="#_footnote_4" title="View footnote.">4</a>]</sup> project with changed resources, Eclipse will invoke
method <code>IncrementalProjectBuilder#build(int,Map,IProgressMonitor)</code> of the project’s <code>XtextBuilder</code><br>
(NOTE: from this point on, we are in the context of a <code>current project</code>)</p>
</li>
<li>
<p>in <code>XtextBuilder#build(int,Map,IProgressMonitor)</code>:<br>
the builder creates an empty instance of <code>ToBeBuilt</code> (Xtext specific)</p>
</li>
<li>
<p>in <code>XtextBuilder#incrementalBuild(IResourceDelta,IProgressMonitor)</code>:</p>
<div class="olist loweralpha">
<ol class="loweralpha" type="a">
<li>
<p>The builder will iterate over all files in the project and for each will notify a <code>ToBeBuiltComputer</code> about the
change (added, updated, or deleted) which can then decide how to update the <code>ToBeBuilt</code> instance,</p>
</li>
<li>
<p>then forwards to <code>#doBuild()</code> .</p>
<div class="paragraph">
<p>Note: if user changes 1..* files in a single project but later more files in other, dependant projects
need to be built, the above step will happen for all projects, but will have an effect only for the first project that contains the actual file changes (i.e. in the standard case of saving a single file <code>ToBeBuilt</code> will always be non-empty for the <code>first</code> project, and always empty for the other, dependant projects; if a <code>Save All</code> is done, <code>ToBeBuilt</code> could be non-empty for later projects as well).</p>
</div>
</li>
</ol>
</div>
</li>
<li>
<p>in <code>XtextBuilder#doBuild(ToBeBuilt,IProgressMonitor,BuildType)</code>:</p>
<div class="olist loweralpha">
<ol class="loweralpha" type="a">
<li>
<p>first check if <code>ToBeBuilt</code> is empty AND global build queue does not contain URIs for current project &#8594; then abort (nothing to do here)</p>
</li>
<li>
<p>creates instance of BuildData with:</p>
<div class="olist lowerroman">
<ol class="lowerroman" type="i">
<li>
<p>name of current project (as string)</p>
</li>
<li>
<p>newly created, fresh <code>ResourceSet</code></p>
</li>
<li>
<p>the <code>ToBeBuilt</code> (containing URIs of actually changed resources within current project, possibly filtered by <code>ToBeBuiltComputer</code>)</p>
</li>
<li>
<p>the <code>QueuedBuildData</code> (an injected singleton)</p>
</li>
<li>
<p>mode flag <code>indexingOnly</code> (only true during crash recovery)</p>
</li>
</ol>
</div>
</li>
<li>
<p>invoke <code>IBuilderState</code> passing the <code>BuildData</code><br>
&#8594; updates itself (it is the global Xtext index) to reflect all changes in <code>current project</code>; validates and updates markers; runs transpiler (see below for details)</p>
</li>
<li>
<p>invoke all registered <code>IXtextBuilderParticipants</code> (Xtext specific) for the <code>current project</code></p>
<div class="ulist">
<ul>
<li>
<p>this is where normally we would do validation and run the transpiler; however, for performance reasons (do not load resource again) we already do this in the <code>IBuilderState</code> (this is the idea of the <code>GenerateImmediatelyBuilderState</code>)</p>
</li>
<li>
<p>in our implementation, almost nothing is done here, except trivial stuff such as deleting files during clean build</p>
<div class="paragraph">
<p>At this point: returning from all methods.</p>
</div>
</li>
</ul>
</div>
</li>
</ol>
</div>
</li>
<li>
<p>back in <code>XtextBuilder#build(int,Map,IProgressMonitor)</code>:<br>
&#8594; return with an array of IProjects; in our case: we return all other N4JSProjects referenced in the package.json of the project</p>
<div class="ulist">
<ul>
<li>
<p>important: these are <strong>not</strong> the projects that will be processed next: we need to continue with projects that depend on the current project, not with projects the current project depends on!</p>
</li>
<li>
<p>Eclipse calls the returned projects <code>interestingProjects</code> and uses that as a hint for further processing; details not discussed here.</p>
</li>
</ul>
</div>
</li>
<li>
<p>continue with step <a href="#itm:start">one</a>:<br>
Eclipse will invoke <code>XtextBuilder#build(int,Map,IProgressMonitor)</code> again for all other projects that have a dependency to the <code>current project</code> of the previous iteration, plus all remaining projects with changed resources.</p>
</li>
</ol>
</div>
</div>
<div class="sect3">
<h4 id="sec:Incremental_Builder_Overview__IBuilderState"><a class="anchor" href="#sec:Incremental_Builder_Overview__IBuilderState"></a><a class="link" href="#sec:Incremental_Builder_Overview__IBuilderState">7.5.2. IBuilderState</a></h4>
<div class="paragraph">
<p>Invoked: once for each project containing a changed resource and dependant projects.<br>
Input: one instance of <code>BuildData</code>, as created by <code>XtextBuilder</code>, containing:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>name of current project (as string)</p>
</li>
<li>
<p>newly created, fresh <code>ResourceSet</code></p>
</li>
<li>
<p>the <code>ToBeBuilt</code></p>
<div class="olist loweralpha">
<ol class="loweralpha" type="a">
<li>
<p>set of to-be-deleted URIs</p>
</li>
<li>
<p>set of to-be-updated URIs</p>
</li>
</ol>
</div>
</li>
<li>
<p>the <code>QueuedBuildData</code>, an injected singleton maintaining the following values <sup class="footnote">[<a id="_footnoteref_5" class="footnote" href="#_footnote_5" title="View footnote.">5</a>]</sup>:</p>
<div class="olist loweralpha">
<ol class="loweralpha" type="a">
<li>
<p>a queue of URIs per project (below called the <code>global queue</code>)<br>
(actually stored in <code>QueuedBuildData#projectNameToChangedResource</code>)</p>
</li>
<li>
<p>a collection of <code>all remaining URIs</code><br>
(derived value: queued URIs of all projects + queues URIs not associated to a project (does not happen in N4JS))</p>
</li>
<li>
<p>a collection of <code>pending deltas</code> (always empty in N4JS; probably only used for interaction with Java resources)</p>
</li>
</ol>
</div>
</li>
<li>
<p>mode flag <code>indexingOnly</code> (only true during crash recovery)</p>
</li>
</ol>
</div>
<div class="sect4">
<h5 id="copy-and-update-xtext-index"><a class="anchor" href="#copy-and-update-xtext-index"></a><a class="link" href="#copy-and-update-xtext-index">7.5.2.1. Copy and Update Xtext Index</a></h5>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>in <code>IBuilderState#update(BuildData,IProgressMonitor)</code>:<br>
creates a copy of its <code>ResourceDescriptionsData</code> called <code>newData</code> <sup class="footnote">[<a id="_footnoteref_6" class="footnote" href="#_footnote_6" title="View footnote.">6</a>]</sup></p>
</li>
<li>
<p>in <code>AbstractBuilderState#doUpdate(&#8230;&#8203;)</code>:<br>
updates <code>newData</code> by writing new resource descriptions into it.</p>
<div class="ulist">
<ul>
<li>
<p>Creates a new load operation (<code>LoadOperation</code>) instance from the <code>BuildData#getToBeUpdated()</code> and loads all entries. While iterating and loading the resource descriptions, it updates <code>newData</code> by registering new resource descriptions that are being created on the fly from the most recent version of the corresponding resources.</p>
</li>
<li>
<p>Adds these resources to the current project’s build queue. (<code>BuildData#queueURI(URI uri)</code>)</p>
</li>
</ul>
</div>
</li>
<li>
<p>for all to-be-deleted URIs given in <code>ToBeBuilt</code> in the <code>BuildData</code>, removes the corresponding <code>IResourceDescription</code> from <code>newData</code></p>
<div class="ulist">
<ul>
<li>
<p><code>ToBeBuilt#getAndRemoveToBeDeleted()</code> returns all URIs that have been marked for deletion but not marked for update and will clear the set of to-be-deleted URIs in <code>ToBeBuilt</code>.</p>
</li>
</ul>
</div>
</li>
</ol>
</div>
</div>
<div class="sect4">
<h5 id="build-state-setup-phase"><a class="anchor" href="#build-state-setup-phase"></a><a class="link" href="#build-state-setup-phase">7.5.2.2. Build State Setup Phase</a></h5>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Calculates a set <code>allRemainingURIs</code> <sup class="footnote">[<a id="_footnoteref_7" class="footnote" href="#_footnote_7" title="View footnote.">7</a>]</sup> as follows:</p>
<div class="ulist">
<ul>
<li>
<p>Initially contains all resource URIs from <code>newData</code>.</p>
</li>
<li>
<p>All URIs will be removed from it that are marked for update (<code>BuildData#getToBeUpdated()</code>).</p>
</li>
<li>
<p>Finally, all URIs will be removed from it that are already queued for build/rebuild. (<code>BuildData#getAllRemainingURIs()</code>).</p>
</li>
</ul>
</div>
</li>
<li>
<p>Creates an empty set <code>allDeltas</code> of resource description deltas<br>
(c.f. <code>IResourceDescription.Delta</code>). <sup class="footnote">[<a id="_footnoteref_8" class="footnote" href="#_footnote_8" title="View footnote.">8</a>]</sup></p>
</li>
<li>
<p><a id="itm:processDeleted"></a> <strong>Process Deleted:</strong> for all to-be-deleted URIs, creates a delta where the old state is the current state of the resource and the new state is <code>null</code> and adds it to <code>allDeltas</code>.</p>
</li>
<li>
<p>Adds all <code>pending deltas</code> from <code>QueuedBuildData</code> to <code>allDeltas</code> (does not apply to N4JS).</p>
</li>
<li>
<p><strong><a id="itm:enqueueAffectedResources"></a>Enqueue affected resources, part 1:</strong> adds to the <code>global queue</code> the URIs of all resources affected by the changes in <code>allDeltas</code>.</p>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title=""></i>
</td>
<td class="content">
For N4JS, <code>allDeltas</code> always seems to be empty at this point, so this does nothing at all.
</td>
</tr>
</table>
</div>
</li>
<li>
<p>Creates an empty set <code>changedDeltas</code> for storing deltas that were modified by the build phase and represent an actual change. Unlike <code>allDeltas</code>, this set contains only those URIs that were processed by the builder - the underlying user data information contains the differences between the old and the new state.</p>
</li>
<li>
<p>Creates a new <code>current queue</code> and adds all URIs from the <code>global queue</code> that belong to the <code>current project</code>.</p>
</li>
</ol>
</div>
</div>
<div class="sect4">
<h5 id="process-queued-uris"><a class="anchor" href="#process-queued-uris"></a><a class="link" href="#process-queued-uris">7.5.2.3. Process Queued URIs</a></h5>
<div class="paragraph">
<p>Processes all elements from the queue until it contains no more elements.</p>
</div>
<div id="itm:loadRes" class="olist arabic">
<ol class="arabic">
<li>
<p>Load the resource for the first/next URI on the current queue</p>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title=""></i>
</td>
<td class="content">
In case of a move, the loaded resource could have a different URI!
</td>
</tr>
</table>
</div>
</li>
<li>
<p>Once the resource has been loaded, it removes its URI from the current queue to ensure it will not be processed again.</p>
</li>
<li>
<p>If the loaded resource is already marked for deletion, stop processing this resource and continue with next URI from the current queue (go to step <a href="#itm:loadRes">Load Res</a>)
<sup class="footnote">[<a id="_footnoteref_9" class="footnote" href="#_footnote_9" title="View footnote.">9</a>]</sup></p>
</li>
<li>
<p>Resolves all lazy cross references in the loaded resource. This will trigger post-processing, including all type inference (c.f. <code>ASTProcessor#processAST(&#8230;&#8203;)</code>).</p>
</li>
<li>
<p>Creates a delta for the loaded resource, including</p>
<div class="olist loweralpha">
<ol class="loweralpha" type="a">
<li>
<p>a resource description based on the new state of the resource, wrapped into the <code>EObject</code>-based resource description (as with the Xtext index persistence in <code>EMFBasedPersister#saveToResource()</code>).</p>
</li>
<li>
<p>a resource description for the same resource with the state before the build process.</p>
</li>
</ol>
</div>
</li>
<li>
<p>Adds this new delta to <code>allDeltas</code> and, if the delta represents a change (according to <code>DefaultResourceDescriptionDelta#internalHasChanges()</code>), also adds it to <code>changedDeltas</code>.</p>
</li>
<li>
<p>Adds the resource description representing the new state, stored in the delta, to <code>newData</code>, i.e. the copied <code>ResourceDescriptionsData</code>, replacing the old resource description of the loaded resource <sup class="footnote">[<a id="_footnoteref_10" class="footnote" href="#_footnote_10" title="View footnote.">10</a>]</sup>.</p>
</li>
<li>
<p>If the current queue is non-empty, go to step <a href="#itm:loadRes">Load Res</a> and continue with the next URI in the current queue.</p>
</li>
</ol>
</div>
</div>
<div class="sect4">
<h5 id="queueing-affected-resources"><a class="anchor" href="#queueing-affected-resources"></a><a class="link" href="#queueing-affected-resources">7.5.2.4. Queueing Affected Resources</a></h5>
<div class="paragraph">
<p>When the current queue contains no more URIs (all have been processed) &#8230;&#8203;</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p><a id="itm:updateGlobalQueue"></a> <strong>Enqueue affected resources, part 2:</strong> add to the global queue URIs for all resources affected by the changes in <code>changedDeltas</code> <sup class="footnote">[<a id="_footnoteref_11" class="footnote" href="#_footnote_11" title="View footnote.">11</a>]</sup>.</p>
</li>
<li>
<p>Returns from <code>#doUpdate()</code>, returning <code>allDeltas</code> (only used for event notification).</p>
</li>
<li>
<p>back in <code>IBuilderState#update(BuildData,IProgressMonitor)</code>:<br>
makes the <code>newData</code> the publicly visible, persistent state of the IBuilderState (i.e. the <code>official</code> Xtext index all other code will see).</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>We now provide some more details on how the global queue is being updated, i.e. steps <a href="#itm:enqueueAffectedResources">Enqueue Affected Resources</a> and <a href="#itm:updateGlobalQueue">Update Global Queue</a>.
Due to the language specific customizations for N4JS, this second resource-enqueuing phase is the trickiest part of the incremental building process and has the largest impact on how other resources will be processed and enqueued at forthcoming builder state phases.</p>
</div>
<div class="ulist">
<ul>
<li>
<p>If <code>allDeltas</code> is empty, nothing to do.</p>
</li>
<li>
<p>If <code>allDeltas</code> contains at least one element, we have to check other affected resources by going through the set of all resource URIs (<code>allRemainingURIs</code>) calculated in in the beginning of the build process.</p>
</li>
<li>
<p>Assume we have at least one element in the <code>allDeltas</code> set, the latter case is true and we must check all elements whether they are affected or not. We simply iterate through the <code>allRemainingURIs</code> set and retrieve the old state of the resource description using the resource URI.</p>
</li>
<li>
<p>Once the resource description with the old state is retrieved, we check if it is affected through the corresponding resource description manager. Since we currently support two languages, we have two different ways for checking whether a resource has changed or not. One for package.json files and the other for the N4JS language related resources.</p>
</li>
<li>
<p>The package.json method is the following: get all project IDs referenced from the <code>candidate</code> package.json and compare it with the container-project name of the package.json files from the <code>deltas</code>. The referenced IDs are the followings:</p>
<div class="ulist">
<ul>
<li>
<p>tested project IDs,</p>
</li>
<li>
<p>implemented project IDs,</p>
</li>
<li>
<p>dependency project IDs,</p>
</li>
<li>
<p>provided runtime library IDs,</p>
</li>
<li>
<p>required runtime library IDs and</p>
</li>
<li>
<p>extended runtime environment ID.</p>
</li>
</ul>
</div>
</li>
<li>
<p>The N4JS method is the following:</p>
<div class="ulist">
<ul>
<li>
<p>We consider only those changed deltas which represent an actual change (<code>IResourceDescription.Delta#haveEObjectDescriptionsChanged()</code>) and have a valid file extension (<code>.n4js</code>, <code>.n4jsd</code> or <code>.js</code>).</p>
</li>
<li>
<p>For each <code>candidate</code>, we calculate the imported FQNs. The imported FQNs includes indirectly imported names besides the directly imported ones. Indirectly imported FQNs are, for instance, the FQNs of all transitively extended super class names of a direct reference.</p>
</li>
<li>
<p>We state that a <code>candidate</code> is affected if there is a dependency (for example name imported by a <code>candidate</code>) to any name exported by the description from a delta. That is, it computes if a candidate (with given <code>importedNames</code>) is affected by a change represented by the description from the delta.</p>
</li>
<li>
<p>If a <code>candidate</code> is affected we have to do an additional dependency check due to the lack of distinct unique FQNs. If a project containing the delta equals with the project contained by the candidate, or if the project containing the candidate has a direct dependency to the project containing the delta, we mark a candidate as affected.</p>
</li>
</ul>
</div>
</li>
<li>
<p>If a candidate was marked as affected, it will be removed from the <code>allRemainingURIs</code> and will be added to the build queue.</p>
</li>
<li>
<p>If a candidate has been removed from the <code>allRemainingURIs</code> and queued for the build, we assume its <code>TModule</code> information stored in the user data is obsolete. To invalidate the obsolete information, we wrap the delta in the custom resource description delta so whenever the <code>TModule</code> information is asked for, it will be missing. We then register this wrapped delta into the copied Xtext index, end the builder state for the actual project then invoke the Xtext builder with the next dependent project.</p>
</li>
</ul>
</div>
</div>
</div>
<div class="sect3">
<h4 id="sec:Incremental_Builder_Overview__Example"><a class="anchor" href="#sec:Incremental_Builder_Overview__Example"></a><a class="link" href="#sec:Incremental_Builder_Overview__Example">7.5.3. Example</a></h4>
<div class="paragraph">
<p>To conclude this section, we briefly describe the state of the above five phases through a simple example. Assume a
workspace with four N4JS projects: <em>P1</em>, <em>P2</em>, <em>P3</em> and <em>PX</em>. Each project has one single module with one single
publicly visible class. Also let’s assume project <em>P2</em> depends on <em>P1</em> and <em>P3</em> depends on <em>P2</em>. Project <em>PX</em> have
no dependencies to other projects. Project <em>P1</em> has a module <em>A.n4js</em> with a class <code>A</code>, project <em>P2</em> has one single
module <em>B.n4js</em>. This module has a public exported class <code>B</code> which extends class <code>A</code>. Furthermore, project <em>P3</em> has
one single module: <em>C.n4js</em>. This module contains one exported public class <code>C</code> which extends <code>B</code>. Finally, project
<em>PX</em> has a module <em>X.n4js</em> containing a class <code>X</code> that has no dependencies to any other classes. The figure below
picture depicts the dependencies between the projects, the modules and the classes as described above.</p>
</div>
<div class="imageblock center">
<div class="content">
<img src="chapters/07_typeindex/images/builderStateExample.png" alt="builderStateExample">
</div>
<div class="title">Figure 18. Builder State Example</div>
</div>
<div class="paragraph">
<p>For the sake of simplification, the table below describes a symbol table for all resources:</p>
</div>
<table class="tableblock frame-all grid-all spread">
<caption class="title">Table 3. Resource Symbol Table</caption>
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">P1/src/A.n4js</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">A</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">P2/src/B.n4js</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">B</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">P3/src/C.n4js</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">C</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">PX/src/X.n4js</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td>
</tr>
</tbody>
</table>
<div class="paragraph">
<p>Let assume auto-build is enabled and the workspace contains no errors and/or warnings. We make one simple modification
and expect one single validation error in class <code>C</code> after the incremental builder finished its processing; we delete the
method <code>foo()</code> from class <code>A</code>.</p>
</div>
<div class="paragraph">
<p>After deleting the method in the editor and saving the editor content, a workspace modification operation will run and
that will trigger an auto-build job. The auto-build job will try to build the container project <em>P1</em> of module <em>A</em>. Since
the project is configured with the Xtext builder command, a builder state update will be performed through the Xtext builder.
Initially, due to an Eclipse resource change event (we literally modify the resource from the editor and save it), the
<code>ToBeBuilt</code> instance wrapped into the <code>BuildData</code> will contain the URI of the module <em>A</em> marked for an update. When updating
the copied index content, module <em>A</em> will be queued for a build. While processing the queued elements for project <em>P1</em>,
module <em>A</em> will be processed and will be added to the <code>allDeltas</code> set. Besides that, it will be added to the <code>changedDeltas</code>
set as well. That is correct, because its <code>TModule</code> information has been changed after deleting the public <code>foo()</code> method.
When queuing affected resources, iterating through the set of <code>allRemainingURIs</code>, we recognize that module <em>B</em> is affected.
That is indeed true; module <em>B</em> imports the qualified name of class <code>A</code> from module <em>A</em> and project <em>P2</em> has a direct
dependency to <em>P1</em>. In this builder state phase, when building project <em>P1</em>, module <em>C</em> is not considered as affected.
Although class <code>C</code> from module <em>C</em> also imports the qualified name of class <code>A</code> from module <em>A</em>, project <em>P3</em> does not
have a direct dependency to project <em>P1</em>. When module <em>B</em> becomes enqueued for a forthcoming build phase, we assume its
<code>TModule</code> information is obsolete. We invalidate this <code>TModule</code> related user data information on the resource description
by wrapping the resource description into a custom implementation (<code>ResourceDescriptionWithoutModuleUserData</code>). Due to this
wrapping the resource description for module <em>B</em> will be marked as changed (<code>IResourceDescription.Delta#haveEObjectDescriptionsChanged()</code>)
whenever the old and current states are being compared.</p>
</div>
<div class="paragraph">
<p>The Eclipse builder will recognize (via <code>IProjectDescription#getDynamicReferences()</code>) that project <em>P2</em> depends on project <em>P1</em>
so the Xtext builder will run for project <em>P2</em> as well. At the previous phase we have enqueued module <em>B</em> for the build.
We will therefore run into a builder state update again. We do not have any resource changes this time, so <code>ToBeBuilt</code> will
be empty. Since <code>ToBeBuilt</code> is empty, we do not have to update the copied Xtext index state before the builder state setup
phase. As the result of the previous builder state, phase module <em>B</em> is already enqueued for a build. When processing <em>B</em>
we register it into the <code>allDeltas</code> set. That happens for each resource being processed by the builder state. But it will be
registered into the <code>changedDeltas</code> because we have previously wrapped module <em>B</em> into a customized resource description delta
to hide its obsolete <code>TModule</code> related user data information. Based on the builder state rules and logic described above,
module <em>C</em> will be marked as an affected resource, will be queued for build and will be wrapped into a customized resource
description delta to hide its <code>TModule</code> related user data information.</p>
</div>
<div class="paragraph">
<p>In the next builder state phase, when building project <em>P3</em>, we apply the same logic as we applied for project <em>P2</em>. The
builder state will process module <em>C</em> and will update the Xtext index state. No additional resources will be found as
affected ones, nothing will be queued for build. The build will terminate, since there were no changed <code>IResource</code> instances
and the build queue is empty.</p>
</div>
<div class="paragraph">
<p>The outcome of the incremental build will be a workspace that contains exactly one validation error. The error will be
associated with module <em>C</em> which was exactly our expectation, however, we have to clarify that transitive <em>C</em> dependency
was built due to wrong reasons. Module <em>C</em> was build because we wrapped module <em>B</em> to hide its user data information and
not because it imports and uses class <code>A</code> from module <em>A</em> which should be the logical and correct reason.</p>
</div>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="dirty-state-handling"><a class="anchor" href="#dirty-state-handling"></a><a class="link" href="#dirty-state-handling">7.6. Dirty state handling</a></h3>
<div class="paragraph">
<p>When two or more (N4)JS files are opened in editors and one of them is changed but without persisting this change the other
open editors should be notified and if this change breaks (or heals) references in one of the other open resources their editors
should updated so that warn and error markers are removed or added accordingly.</p>
</div>
<div class="paragraph">
<p>When there are changes in the currently open editor these changes are propagated to all other open editors. Each Xtext editor has
got its own resource set. The <code>N4JSUpdateEditorStateJob</code> runs for each open editor different from the editor where the changes have
been made. In those editors the affected resources are unloaded and removed from the resource set. Then the Xtext resource of
these editors is reparsed. After reparsing scoping and linking is invoked again, but now the references resources are rebuild
as <code>EObjectDescription</code>s. The <code>N4JSResource</code> holds its own content that only contains 1..n slots when proxified.
<code>N4JSTypeScope.getSingleElement</code> (called when resolving cross references and the linked element should be returned) will return the
<code>EObjectDescription</code> created from the <code>ModuleAwareContentsList</code> in <code>N4JSResource</code>, that contains the first slot as proxy and the other
slots as types. <a href="#fig:dirty_state_handling1">Sequence Diagram: Dirty State, Trigger <code>N4JSUpdateEditorStateJob</code></a> shows the flow to trigger the <code>N4JSUpdateEditorStateJob</code> and <a href="#fig:dirty_state_handling2">Sequence Diagram: Dirty State, <code>N4JSUpdateEditorStateJob</code> in Detail</a>
shows the sequence logic of the <code>N4JSUpdateEditorStateJob</code> in detail.</p>
</div>
<div id="fig:dirty_state_handling1" class="imageblock center">
<div class="content">
<img src="chapters/07_typeindex/images/dirty_state_handling1.png" alt="dirty state handling1">
</div>
<div class="title">Figure 19. Sequence Diagram: Dirty State, Trigger <code>N4JSUpdateEditorStateJob</code></div>
</div>
<div id="fig:dirty_state_handling2" class="imageblock center">
<div class="content">
<img src="chapters/07_typeindex/images/dirty_state_handling2.png" alt="dirty state handling2">
</div>
<div class="title">Figure 20. Sequence Diagram: Dirty State, <code>N4JSUpdateEditorStateJob</code> in Detail</div>
</div>
<div class="paragraph">
<p>A concrete example should illustrate the behaviour of the dirty state handling in conjunction with fully and partial loading
of resources:</p>
</div>
<div class="paragraph">
<p>Let A.js as above, and B.js as follows:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-n4js" data-lang="n4js">import A from "A.js"
export class B {}</code></pre>
</div>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>assume is opened and loaded: is created with</p>
<div class="ulist">
<ul>
<li>
<p>is filled with a special proxy to resolve the <a href="#AC">AST</a> of A only if needed.</p>
</li>
<li>
<p>will be set to type A, loaded from <code>EObjectDescription</code> of A.js/A</p>
</li>
</ul>
</div>
</li>
<li>
<p><a href="#AC">AST</a> of A.js is to be accessed, e.g., for displaying JSDoc. A.js is not opened in an editor! is modified as follows:</p>
<div class="ulist">
<ul>
<li>
<p>is filled with <a href="#AC">AST</a>, i.e. proxy in 0 is resolved</p>
</li>
<li>
<p>is updated with parsed type: 1) proxify , 2) unload (remove from content), 3) reload with parsed types again</p>
</li>
</ul>
</div>
</li>
<li>
<p>Assume now that A.js is opened and edited by the user.</p>
<div class="ulist">
<ul>
<li>
<p>Reconceiler replaces with modified <a href="#AC">AST</a></p>
</li>
<li>
<p>LazyLinker updates </p>
</li>
<li>
<p>is proxified</p>
</li>
<li>
<p>B <code>searches</code> for A and finds updated </p>
</li>
</ul>
</div>
</li>
</ol>
</div>
<div class="paragraph">
<p><em>Each opened Xtext editor has got its own resource set!</em> Such a resource set contains the resource for the currently edited
file in the first place. When starting editing the file, the resource is reparsed , reloaded and linking (resolving the cross
references) is done. By resolving the cross references <code>N4JSTypeScope</code> is used and now the URIs of the linked elements are belonging
to resources not contained in the resource set of the editor so these resources a create in the resource set and their contents
is loaded from the resource descriptions via
<code>N4JSResource.loadFromDescription</code> .</p>
</div>
<div class="paragraph">
<p>When the resource content is loaded from the existing resource description available from Xtext index the first slot is set to be
a proxy with name <code>#:astProxy</code>.
After that for all exported EObject descriptions of that resource description the user data is fetched and deserialized to types
and these types are added to the slots of the resource in order they were exported. But the order is not that important anymore.</p>
</div>
<div class="paragraph">
<p>As the resource set for the editor is configured to use the DirtyStateManager ( <code>ExternalContentSupport.configureResourceSet(resourceSet, dirtyStateManager)</code> ),
all other open editors will be notified by changes in the current editor. This is done by <code>N4JSDirtyStateEditorSupport</code> that schedules
a <code>N4JSUpdateEditorStateJob</code> that create a new resource description change event.</p>
</div>
<div class="paragraph">
<p>Via <code>isAffected</code> and <code>ResourceDescription.getImportedNames</code> it is determine if a change in another resource affects this resource.</p>
</div>
<div class="paragraph">
<p>Before loading the resource always <code>N4JSDerivedStateComputer.installDerivedState</code> is execute that, as we learned earlier, is responsible
for creating the types in the resource.</p>
</div>
<div class="paragraph">
<p>On every change to a N4JS file that requires a reparse the <code>N4JSDerivedStateComputer.discardDerivedState</code> is executed. This method do an
unload on every root element at the positions 1 to n. In the <code>N4JSUnloader</code> all contents of the this root elements are proxified (i.e.
there is a proxy URI set to each of them) and the references to the <a href="#AC">AST</a> are set to null (to avoid notifications causing
concurrent model changes). The proxification indicates for all callers of these elements, that they have to reload them. Afterwards
it discards the complete content of the resource. The content is build up again with the reparse of the N4JS file content.</p>
</div>
<div class="paragraph">
<p>As each editor has its own resource set, only the resource belonging to the current editor is fully loaded. All other referenced
resources are only partially loaded, i.e. only the slot 1 of these resources are loaded (i.e. the types model elements) in this
resource set. Linking is done only against these types model elements. Synchronization between the resource sets of multiple open
editors is done via update job as described above.</p>
</div>
<div class="sect3">
<h4 id="use-case-restoring-types-from-user-data"><a class="anchor" href="#use-case-restoring-types-from-user-data"></a><a class="link" href="#use-case-restoring-types-from-user-data">7.6.1. Use case: Restoring types from user data</a></h4>
<div class="ulist">
<ul>
<li>
<p>Use case: referencing resources in editor: This has been described already in context of dirty state handling</p>
</li>
<li>
<p>Use case: referencing resources from JAR: This is still to be implemented.% taskIDE-37</p>
</li>
</ul>
</div>
</div>
<div class="sect3">
<h4 id="use-case-updating-the-xtext-index"><a class="anchor" href="#use-case-updating-the-xtext-index"></a><a class="link" href="#use-case-updating-the-xtext-index">7.6.2. Use case: Updating the Xtext index</a></h4>
<div class="paragraph">
<p>When a N4JS file is changed in way that requires reparsing the file, the underlying resource is completely unloaded and loaded again.
By this the also the elements at the Xtext index are recreated again, belonging to this resource (i.e. new entries for new elements
in the resource, update index elements of changed elements, delete index entries for deleted elements).</p>
</div>
<div class="paragraph">
<p>When Eclipse is closed the Xtext index is serialized in a file.</p>
</div>
<div class="imageblock center">
<div class="content">
<img src="chapters/07_typeindex/images/index_serialization.png" alt="index serialization">
</div>
</div>
<div class="paragraph">
<p>When starting Eclipse again, the Xtext index is restored from this file:</p>
</div>
<div class="imageblock center">
<div class="content">
<img src="chapters/07_typeindex/images/loading_existing_index.png" alt="loading existing index">
</div>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_project-model"><a class="anchor" href="#_project-model"></a><a class="link" href="#_project-model">8. Project Model</a></h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="sec:Package_json"><a class="anchor" href="#sec:Package_json"></a><a class="link" href="#sec:Package_json">8.1. Package.json File</a></h3>
<div class="paragraph">
<p>See [<a href="#N4JSSpec">N4JSSpec</a>] for the format specification of N4JS-specific package.json files.
Based on the JSON-model-based AST that can be parsed from the package.json file, we transform the information that can be extracted into an instance of the N4JS-specific ProjectDescription model.
This model is defined by means of EMF, the Xcore file is found in the N4JS model bundle.</p>
</div>
<div id="fig:projectDescriptionModel" class="imageblock">
<div class="content">
<img src="chapters/08_projectModel/images/cd_projectDescription.svg" alt="Project Description Model">
</div>
</div>
</div>
<div class="sect2">
<h3 id="_accessing-project-information"><a class="anchor" href="#_accessing-project-information"></a><a class="link" href="#_accessing-project-information">8.2. Accessing Project Information</a></h3>
<div class="paragraph">
<p>The information in the package.json files is parsed into memory at runtime, e.g. within Eclipse or in headless mode. It is made available via the <code>IN4JSCore</code> facade that provides a high-level abstraction when working with the project structure. This facade has two implementations:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>One implementation is backed by the file system, e.g. <code>java.io.File</code> and used in a headless environment</p>
</li>
<li>
<p>Another implementation uses the Eclipse resource model (<code>IProject</code>, <code>IFolder</code>, <code>IFile</code>) to describe the contents of an <code>IN4JSProject</code>. This implementation is automatically kept in sync whenever the contents of a package.json file is edited by a user. It is also maintained as dynamic project information to make the Eclipse workspace aware of the declared dependencies. That is, Eclipse projects know about the project references in the package.json file.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>The project model is mimicing the handle based services of Eclipse’s <code>JavaCore</code>, the handles are often represented as EMF <code>URIs</code> though. Therefore the API allows to retrieve a source folder for a given N4JS resource, all the libraries that are configured for a project or simply the project’s name or the information if it exists. Subsequent components in the processing chain like the scope provider can leverage the API to deduce the container configuration and visiblity constraints. Project references are resolved transparently.</p>
</div>
<div class="paragraph">
<p>The Xtext index participation is implemented by means of the <code>N4JSToBeBuiltComputer</code> and the <code>N4JSAllContainersState</code>. These classes provide access to the container configuration.</p>
</div>
<div class="paragraph">
<p>The precedence for the dependency resolution follows this lookup chain:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>An accessible <code>IN4JSProject</code> with the given name is located in the workspace.</p>
</li>
<li>
<p>The library manager provides access to the requested project.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Accordingly, the type lookup follows the same chain due to the container configuration that is deduced from the package.json configuration. Of course is prefers locally defined type that can be found in the current project over types that are located in referenced projects.</p>
</div>
<div class="sect3">
<h4 id="sec:IN4JSCore"><a class="anchor" href="#sec:IN4JSCore"></a><a class="link" href="#sec:IN4JSCore">8.2.1. IN4JSCore</a></h4>
<div class="paragraph">
<p>Facade analogous to org.eclipse.jdt.core.JavaCore.
It is used look up the project or source container where a resource URI belongs to.
It also provides some helper methods to retrieve information from the package.json file.
N4JSRuntimeCore uses the file system and thus uses java.io to access files and folders.
N4JSEclipseCore uses the Eclipse workspace and thus uses org.eclipse.resources API.
Both Core implementations act as wrapper for N4JSModel resp. EclipseN4JSModel.
Instances of the <code>IN4JSCore</code> are obtained as usually via dependency injection, e.g. <code>@Inject IN4JSCore n4jsCore</code>.</p>
</div>
</div>
<div class="sect3">
<h4 id="sec:N4JSModel"><a class="anchor" href="#sec:N4JSModel"></a><a class="link" href="#sec:N4JSModel">8.2.2. N4JSModel</a></h4>
<div class="paragraph">
<p>N4JSModel uses FileBasedWorkspace to load and access the project description from the package.json file and create wrappers for projects (N4JSProject) and source containers (N4JSSourceContainer).</p>
</div>
<div class="paragraph">
<p>A source container is a wrapper for a file system that has been marked as source folder in the package.json file.
For determination of the current project a given EMF URI pointing to the project path is used.
In N4JSModel this location is directly wrapped in N4JSProject.
In N4JSModel a given EMF URI is resolved to a source container by using the specified relative source paths from the package.json file and file system based project location.
If the EMF URI converted to a file URI starts with the absolute source folder path a N4JSProjectSourceContainer is created and returned for that EMF URI.</p>
</div>
<div class="paragraph">
<p>In EclipseN4JSModel (that extends N4JSModel) the last segment of the URI is assumed to be the project name and via the EclipseBasedN4JSWorkspace that wraps the Eclipse workspace a project with that name is tried to be resolved.
This IProject is than wrapped in N4JSEclipseProject.
In EclipseN4JSModel a given EMF URI is resolved to an org.eclipse.core.resources.IResource belonging to the IWorkspaceRoot.
That resource is wrapped in EclipseSourceContainer.</p>
</div>
<div class="paragraph">
<p>N4JSModel is also used to retrieve project dependencies wrapped as N4JSProject.</p>
</div>
</div>
<div class="sect3">
<h4 id="sec:N4JSWorkspace"><a class="anchor" href="#sec:N4JSWorkspace"></a><a class="link" href="#sec:N4JSWorkspace">8.2.3. N4JSWorkspace</a></h4>
<div class="paragraph">
<p>The FileBasedWorkspace and EclipseBasedN4JSWorkspace should only be accessed by N4JSModel resp. EclipseN4JSModel as they know the contract for the URI scheme.
The FileBasedWorkspace creates AbstractTreeIterator for the direct contents of a source container, so that their children can be navigated by this as well.
It then filters out all directories and returns an iterator of all files as EMF URIs.</p>
</div>
<div class="paragraph">
<p>The EclipseBasedN4JSWorkspace wraps the IWorkspaceRoot.</p>
</div>
<div class="paragraph">
<p>Fetching the project description read out from the package.json file is cached in both workspace implementations.
In FileBasedWorkspace the LazyProjectDescriptionHandle is used as proxy and in EclipseBasedN4JSWorkspace the ProjectDescriptionLoadListener is used to invalidate the cache when the package.json file has been changed.
ProjectDescriptionLoadListener also ensures that dependent projects are considered by the Eclipse builder by setting dynamic dependencies in the project meta data.
It also updates these project dependencies if it is required and recalculates all source containers.</p>
</div>
<div class="paragraph">
<p>In the tests another implementation, MockWorkspace, is used, that provides a dummy project description.</p>
</div>
</div>
<div class="sect3">
<h4 id="sec:N4JSProject"><a class="anchor" href="#sec:N4JSProject"></a><a class="link" href="#sec:N4JSProject">8.2.4. N4JSProject</a></h4>
<div class="paragraph">
<p>A N4JSProject represents a configured project as defined in the package.json file.
So in principles it wraps access to N4JSModel (and so to containing source containers, libraries and so on)
and to some information from the package.json file directly (like defined module filters, vendorId and others).
It is also used to compare depending projects.</p>
</div>
<div class="paragraph">
<p>N4JSProject is the runtime representation while N4JSEclipseProject represents a project in the Eclipse workspace.
N4JSProject is identified by its EMF location URI while N4JSEclipseProject wraps the underlying org.eclipse.core.resources.IProject.</p>
</div>
<div class="paragraph">
<p>In tests MockProject is used.</p>
</div>
</div>
<div class="sect3">
<h4 id="sec:SourceContainer"><a class="anchor" href="#sec:SourceContainer"></a><a class="link" href="#sec:SourceContainer">8.2.5. SourceContainer</a></h4>
<div class="paragraph">
<p>A source container contains either source files for production, tests or external declarations.
By default all these files resp. their containing types will be exported to the Xtext index.
A source container belongs exactly to one project it is identified by its project relative location.</p>
</div>
<div class="paragraph">
<p>A N4JSProjectSourceContainer is a container that contains unpacked n4js, js and n4jsd files that can be modified.
By default all these files are syntactically and semantically validated (this can be configured by the use of module filters).
Except for n4jsd files, all files are compiled by the configured compilers.</p>
</div>
<div class="paragraph">
<p>EclipseSourceContainer specializes N4JSProjectSourceContainer to work on top of the Eclipse resources API.
Thus besides the location it also wraps the IFolder.</p>
</div>
<div class="paragraph">
<p>The IFile was chosen instead of using the java.io.File because changes to an IFile (and IResource in general) trigger a workspace change event so that the Xtext builder is triggered properly.</p>
</div>
</div>
<div class="sect3">
<h4 id="sec:N4JSProjectsStateHelper"><a class="anchor" href="#sec:N4JSProjectsStateHelper"></a><a class="link" href="#sec:N4JSProjectsStateHelper">8.2.6. N4JSProjectsStateHelper</a></h4>
<div class="paragraph">
<p>Calculates the visible containers for a given container, where containers are source containers within the project as well as source containers of other depending projects in workspace.
The calculated handles are cached and invalidated if the project description file has changed or the project has been closed or reopened.</p>
</div>
</div>
</div>
<div class="sect2">
<h3 id="sec:Caching"><a class="anchor" href="#sec:Caching"></a><a class="link" href="#sec:Caching">8.3. Caching</a></h3>
<div class="paragraph">
<p>Caching is heavily used in the ExternalLibraryWorkspace and the N4JSProjectsStateHelper.
The ExternalLibraryWorkspace relies on caching to provide data about all installed npms, their locations, names, shadowing, dependencies and so on.
The caching of the ExternalLibraryWorkspace is implemented in the ExternalProjectMappings which inspects all external locations and builds all necessary mappings.
The set of mappings start from a simple list of all npms, include mappings that map from location or name to a N4JSExternalProject, or give reduced set of projects.</p>
</div>
<div class="sect3">
<h4 id="_caching-of-externallibraryworkspace"><a class="anchor" href="#_caching-of-externallibraryworkspace"></a><a class="link" href="#_caching-of-externallibraryworkspace">8.3.1. Caching of ExternalLibraryWorkspace</a></h4>
<div class="paragraph">
<p>A reduced set of projects is used since not all npms are actually necessary projects for the N4JS IDE.
Most transitively installed plain-JS npms are of no interest since they are completely invisible to the user.
The reduced set of projects always consists of all user workspace projects and all shipped libraries.
From the set of all installed npms, only those are necessary that are dependencies of a non-plain-JS projects.
Shadowed projects are also not included in the reduced set of npms.</p>
</div>
<div class="paragraph">
<p>To access projects that are not included in the reduced set of npms, the ExternalProjectMappings provides some collections that contain complete set of npms.
Additionally, some mappings also provide information about not necessary npms.
Note that mappings that use the project name as a key naturally cannot provide information about shadowed projects.</p>
</div>
<div class="paragraph">
<p>The mapping cache is updated every time a refresh is triggered (e.g. at startup or by hitting F5).
Also, every action of the library manager (such as installing or registering npms) triggers a refresh.</p>
</div>
</div>
<div class="sect3">
<h4 id="_caching-of-n4jsprojectsstatehelper"><a class="anchor" href="#_caching-of-n4jsprojectsstatehelper"></a><a class="link" href="#_caching-of-n4jsprojectsstatehelper">8.3.2. Caching of N4JSProjectsStateHelper</a></h4>
<div class="paragraph">
<p>The N4JSProjectsStateHelper uses the MultiCleartriggerCache for caching information about projects of the user workspace.
The EclipseBasedN4JSWorkspace does not caching at all, but provides information about project descriptions which is expensive to compute on the fly.
Hence this information is cached in the MultiCleartriggerCache and updated every time a project description changes, is added or removed.</p>
</div>
<div class="paragraph">
<p>Sometimes, a project description is rendered invalid as a side effect of a change on another project description.
In this case, the cache of both of project descriptions has to be updated.
The implementation to cope with these side effects uses the MultiCleartriggerCache which allows to set multiple triggers that will clear a cached object.</p>
</div>
<div class="paragraph">
<p>However, it seems reasonable to align the caching of the user workspace to the caching of the external workspace.
The reason is that caching of user workspace information such as N4JSProjects would increase build performance significantly.
This is since as of now, projects (and information about all their source containers) is computed on the fly, that causes thousands of expensive calls to the file system.</p>
</div>
</div>
</div>
<div class="sect2">
<h3 id="sec:WildcardPathFilter"><a class="anchor" href="#sec:WildcardPathFilter"></a><a class="link" href="#sec:WildcardPathFilter">8.4. WildcardPathFilter</a></h3>
<div class="paragraph">
<p>This class encapsulates the logic to resolve (wildcard containing) paths against the file system.
With the method matchPath it is also possible to resolve a path without using the file system.
This class is also able to resolve relative navigation in paths.</p>
</div>
</div>
<div class="sect2">
<h3 id="sec:ProjectUtils"><a class="anchor" href="#sec:ProjectUtils"></a><a class="link" href="#sec:ProjectUtils">8.5. ProjectUtils</a></h3>
<div class="paragraph">
<p>ProjectUtils provides additional methods for providing information only required in compilation, e.g. like file and module descriptor.
It uses IN4JSCore to retrieve the information of output path and whether module wrapping is required for a given file.</p>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_binding"><a class="anchor" href="#_binding"></a><a class="link" href="#_binding">9. Binding</a></h2>
<div class="sectionbody">
<div class="admonitionblock warning">
<table>
<tr>
<td class="icon">
<i class="fa icon-warning" title=""></i>
</td>
<td class="content">
This section may be outdated!
</td>
</tr>
</table>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Binding_Design_Rationale"><a class="anchor" href="#sec:Binding_Design_Rationale"></a><a class="link" href="#sec:Binding_Design_Rationale">9.1. Design Rationale</a></h3>
<div class="paragraph">
<p>Binding references to declarations follows the Xtext mechanism based on local <code>N4JSScopeProvider</code> and a global <code>N4JSGlobalScopeProvider</code> scope providers. The basic question is: to which elements are references bound to. This in particular interesting for all kind of type declarations, including functions as they are interpreted as types. These declarations are Janus-faced: On the one side, they are targets of type references as <code>Type</code>, and on the other side they can also be target of identifier references bound to some so called <code>IdentifiableElement</code>. As explained in <a href="#_type_index">[_type_index]</a>, special type objects (<code>TClass</code> etc.) are created from the original declarations. These type objects are used as targets for both kind of references. The following table summarizes the reference-target relations relevant for N4JS (not the standalone type grammar).</p>
</div>
<table class="tableblock frame-all grid-all spread">
<caption class="title">Table 4. N4JS Cross References</caption>
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Reference</th>
<th class="tableblock halign-left valign-top">Target Type</th>
</tr>
</thead>
<tbody>
<tr>
<th class="tableblock halign-left valign-top" colspan="2"><p class="tableblock">N4JS</p></th>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">ImportDeclaration.importedModule</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">TModule</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">NamedImportSpecifier.importedElement</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">types.IdentifiableElement</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">IdentifierRef.id</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">types.IdentifiableElement</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">ParameterizedPropertyAccessExpression.property</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">types.TMethod</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">PropertyAccessExpression.property</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">types.TMember</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">N4Getter/N4SetterDeclaration.field</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">N4FieldDeclaration</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Continue/Break-Statement.label</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">LabelledStatement</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock">Type Expressions</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">ParameterizedTypeRef.declaredType</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Type</p></td>
</tr>
</tbody>
</table>
<div class="paragraph">
<p><a href="#fig:cd_scoping">Overview Scoping Package</a> gives an overview over the most important classes in the scoping package, with the <code>N4JSScopeProvider</code> and the used customized scopes created by the scope providers.</p>
</div>
<div id="fig:cd_scoping" class="imageblock center">
<div class="content">
<img src="chapters/09_binding/images/cd_scoping.svg" alt="cd scoping">
</div>
<div class="title">Figure 21. Overview Scoping Package</div>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Binding_to_Members"><a class="anchor" href="#sec:Binding_to_Members"></a><a class="link" href="#sec:Binding_to_Members">9.2. Binding to Members</a></h3>
<div class="paragraph">
<p>Members of different types, such as classes and also record types or enumerations, are bound using the <code>MemberScopeProvider</code>. This often returns a <code>MemberScope</code>, which directly works on the members. Most types with members are implemented by subclasses of <code>ContainerType</code>, using <code>CollectMembersHelper</code> to collect all members and <code>FindMemberHelper</code> for retrieving a member by its name via <code>ContainerTypes</code>. Ensure that when types with members are added to override appropriate methods in all of these related classes (e.g., <code>CollectMembersHelper</code>, <code>AbstractHierachyTraverser</code> and <code>FindMemberHelper</code> uses polymorphic dispatch to handle different subtypes – so you won’t be able to find a member if you do not adjust these helpers).</p>
</div>
</div>
<div class="sect2">
<h3 id="sec:Binding_Getter_Setter"><a class="anchor" href="#sec:Binding_Getter_Setter"></a><a class="link" href="#sec:Binding_Getter_Setter">9.3. Getter / Setter Binding</a></h3>
<div class="paragraph">
<p>For customized binding of getters / setters, see <a href="#sec:Field_Accessors">Accessors</a>.</p>
</div>
</div>
<div class="sect2">
<h3 id="chap:Statics"><a class="anchor" href="#chap:Statics"></a><a class="link" href="#chap:Statics">9.4. Static Member Binding</a></h3>
<div class="paragraph">
<p>For customized binding of static members, see <a href="#sec:Static_Members">Static Members</a>.</p>
</div>
</div>
<div class="sect2">
<h3 id="sec:Binding_Enumeration"><a class="anchor" href="#sec:Binding_Enumeration"></a><a class="link" href="#sec:Binding_Enumeration">9.5. Enumeration Literals Binding</a></h3>
<div class="ulist">
<ul>
<li>
<p>introduced new type ref EnumTypeRef: it behaves comparable to ClassifierTypeRef, but with the difference that the MemberScopeProvider filters for a given EnumTypeRef filters all literals of the contained TEnum (in comparison the MemberScopeProvider filters for a given ClassifierTypeRef all static members of the contained classifier)</p>
</li>
<li>
<p>it isn’t possible to access literals on a enumeration literal itself, although this literal is typed as TEnum (that contains TEnumLiterals)</p>
</li>
<li>
<p>as there are currently no additional fields and operations for enumeration literals defined (in Java there is name and value()), the scope for literals is currently empty</p>
</li>
</ul>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Accessibility_of_types_and_members"><a class="anchor" href="#sec:Accessibility_of_types_and_members"></a><a class="link" href="#sec:Accessibility_of_types_and_members">9.6. Accessibility of types and members</a></h3>
<div class="paragraph">
<p>Member access and type access has to be constrained and validated against the accessibility rules of N4JS. Therefore, the scoping annotates certain elements as erroneous to detect invalid references.</p>
</div>
<div class="paragraph">
<p>Basically two different approaches are used to implement that behavior:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>The <code>VisibilityAwareTypeScope</code> and <code>VisibilityAwareMemberScope</code> decorate an existing scope to validate the result on access. This allows to lazily check the visibility of the returned element. If it is not accessible, it is wrapped in a <code>AbstractDescriptionWithError</code> which will be indentified as such by the <code>ErrorAwareLinkingService</code>. Before the binding is resolved and the EMF proxy is replaced, the error message is used to create an EMF diagnostic.</p>
</li>
<li>
<p>For other cases, the scopes are produced differently, e.g. if all elements are easily enumerable and have to be collected before the scope is created (e.g. for imported elements), the scoped elements are validated eagerly to put them into the correct layer of scopes. That is, the valid descriptions may shadow the invalid description. Since there are more error conditions for these cases, e.g. duplicate imports and similar cases, the accessibility is checked before the concrete member is accessed. All the instances <code>AbstractDescriptionWithError</code> are put into the <code>MapBasedScope</code> immediately.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>In that sense, accessibility checks are basically implemented as decorators for the scoping itself. Bindings are established but flagged as errors.</p>
</div>
<div class="paragraph">
<p>Default visibility of members is calculated in <code>Types.xcore</code> (in <code>getTypeAccessModifier</code> and <code>getMemberAccessModifier</code> etc.). Visibility is checked in <code>org.eclipse.n4js.scoping.accessModifiers.MemberVisibilityChecker</code> and validators.</p>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Member_Scope_Example"><a class="anchor" href="#sec:Member_Scope_Example"></a><a class="link" href="#sec:Member_Scope_Example">9.7. Member Scope Example</a></h3>
<div class="paragraph">
<p>In this section, we are going to have a look at the creation process of <code>MemberScope</code>.</p>
</div>
<div class="listingblock">
<div class="title">C.n4js</div>
<div class="content">
<pre class="highlight"><code>export public class C {
private m1: int;
public m2: int;
}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="title">Test.n4js</div>
<div class="content">
<pre class="highlight"><code>import { C } from "C";
let c: C = new C();
c.m1; // Error -&gt; The field m1 is not visible
c.m2; // OK -&gt; m2 is visible at this context</code></pre>
</div>
</div>
<div class="paragraph">
<p>Assume that we need to figure out to which element the <code>ParameterizedPropertyAccessExpression c.m1</code> in the <code>ExpressionStatement c.m1</code> binds to. To answer this question, <code>N4JSScopeProvider.getScope(context, reference)</code> is triggered whereby <code>context</code> is the <code>ParameterizedPropertyAccessExpression</code> and <code>reference</code> is <code>EReference property</code> (<code>property</code> is the cross-reference element defined in <code>ParameterizedPropertyAccessExpression</code> 's grammar).</p>
</div>
<div class="paragraph">
<p><code>N4JSScopeProvider.getScope(context, reference)</code> does not implement the scoping but delegates to corresponding methods based on the type of <code>context</code>. In our example, since <code>context</code> is a <code>ParameterizedPropertyAccessExpression</code>, the scoping logic is delegated to the method that creates a <strong>MemberScope</strong> for the context <code>ParameterizedPropertyAccessExpression c.m1</code> based on the receiver type of <code>c</code> which is class <code>C</code>.
The resulting scope instance returned by <code>N4JSScopeProvider.getScope()</code> in our example is of type <code>TypingStrategyAwareMemberScope</code> as shown in <a href="#fig:memberscope-example">Member scope hierarchy</a> .</p>
</div>
<div id="fig:memberscope-example" class="imageblock" style="text-align: center">
<div class="content">
<img src="chapters/09_binding/images/memberscope_example.svg" alt="memberscope example">
</div>
<div class="title">Figure 22. Member scope hierarchy</div>
</div>
<div class="paragraph">
<p>In the hierarchy, the top-level scope is the NULL scope. Directly below the NULL scope is a MemberScope which contains all members of <code>N4Object</code> since the class <code>C</code> implicitly inherits <code>N4Object</code>. The other <code>MemberScope</code> instance beneath contains all members of the class <code>C</code> <strong>regardless of their visibility</strong>. These members are <code>m1</code> and <code>m2</code>. While <code>m2</code> is can be accessed by <code>c.m2</code>, <code>m1</code> it not visible at <code>c.m1</code>. The <code>VisibilityAwareMemberScope</code> implements precisely this behavior. In particular, it returns all members of <code>C</code> that are visible at the current <code>context</code> (here the element <code>m2</code>), while wrapping non-visible members (here the element <code>m</code>) in <code>InvisibleMemberDescription</code> instances. These <code>InvisibleMemberDescription</code> instances of type <code>IEObjectDescriptionWithError</code> contain issue code and error message related to accessibility problems and are recognized during the error-aware linking phase done by <code>ErrorAwareLinkingService</code>. It is worth to emphasize the motivation behind use of <code>IEObjectDescriptionWithError</code> is to provide more informative error messages to the user other than <em>Cannot reference element&#8230;&#8203;</em> Another example of <code>IEObjectDescriptionWithError</code> is <code>WrongWriteAccessDescription</code> that is used when we, try to write to a getter and no corresponding setter exists.</p>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Scoping_for_Members_of_Composed_Type_Explained"><a class="anchor" href="#sec:Scoping_for_Members_of_Composed_Type_Explained"></a><a class="link" href="#sec:Scoping_for_Members_of_Composed_Type_Explained">9.8. Scoping for Members of Composed Type (Union/Intersection) Example</a></h3>
<div class="paragraph">
<p>In this section, we will have a look at how scoping is implemented for composed type, i.e. union or intersection type with an example of union type. Intersection is done similarly. Before reading this, it is strongly recommended to read <a href="#sec:Member_Scope_Example">Member Scope Example</a> first.</p>
</div>
<div class="listingblock">
<div class="title">Defs.n4js</div>
<div class="content">
<pre class="highlight"><code>export public class C {
private m1: int;
public m2: int;
}
export public class D {
private m1: int;
get m2(): int { return 42; };
}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="title">Test.n4js</div>
<div class="content">
<pre class="highlight"><code>import { C, D } from "Defs";
let cud : C|D;
cud.m2 = 10;</code></pre>
</div>
</div>
<div class="paragraph">
<p>Assume that we need to find out to what element the <code>ParameterizedPropertyAccessExpression cud.m2</code> in the <code>ExpressionStatement cud.m2</code> binds to.
This is a question for scoping. Since the receiver type of <code>cud</code> is a union type <code>C|D</code>, a <code>UnionMemberScope</code> is created that contains two subscopes, each of which corresponds to an individual type in the union. The resulting scope hierarchy is graphically depicted in <a href="#fig:unionmemberscope-example">Union member scope hierarchy</a>.</p>
</div>
<div id="fig:unionmemberscope-example" class="imageblock" style="text-align: center">
<div class="content">
<img src="chapters/09_binding/images/unionmemberscope_example.svg" alt="unionmemberscope example">
</div>
<div class="title">Figure 23. Union member scope hierarchy</div>
</div>
<div class="paragraph">
<p>The two subscopes are of type <code>TypingStrategyAwareMemberScope</code> and created exactly the same way as described in <a href="#sec:Member_Scope_Example">Member Scope Example</a>.
The <code>UnionMemberScope</code> instance contains a list of subscopes for all types involved in the union and is responsible for constructing an <code>IEObjectDescription</code> instance for <code>m2</code> by merging all members of the name <code>m2</code> found in all subscopes.
Merging members requires considering a variety of combinations (fields, setters getters, optional/variadic parameters etc.) and thus can become very complicated. To reduce the complexity, the recently refactored implementation splits the proccess into three separate steps.</p>
</div>
<div class="paragraph">
<p>Step 1: Collect information</p>
</div>
<div class="paragraph">
<p>During this phase, members with the name <code>m2</code> are looked up in each subscope and collected into an <code>ComposedMemberInfo</code> instance by <code>ComposedMemberInfoBuilder</code>.
The first subscope (left branch in the <a href="#fig:unionmemberscope-example">Union member scope hierarchy</a>) returns an <code>EObjectDescription</code> wrapping the <code>TField m2</code> of class <code>C</code> and hence <code>TField m2</code> is added to the <code>ComposedMemberInfo</code> instance. The second subscope (right branch in the <a href="#fig:unionmemberscope-example">Union member scope hierarchy</a>) returns a <code>WrongWriteAccessDescription</code> wrapping the <code>TGetter m2</code> of class <code>D</code> and hence <code>TGetter m2</code> is added to <code>ComposedMemberInfo</code> instance. The reason for <code>WrongWriteAccessDescription</code> because <code>cud.m2</code> is trying to write to the getter of the same name in <code>D</code>.</p>
</div>
<div class="paragraph">
<p>At the end of this step, two members <code>public TField m2: int</code> and <code>project TGetter m2(): int</code> are added to <code>ComposedMemberInfo</code>.</p>
</div>
<div class="paragraph">
<p>Step 2: Merge members</p>
</div>
<div class="paragraph">
<p>This phase merges members of the same name into a composed member based on the information about these members collected in Step 1. Note that merge rules can become quite complicated as many situations must be considered. Sometimes, it is not possible to merge at all. If the merge is possible, we need to consider the following properties, among others,</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Member kind: what kind of member is the merge result. For instance, what do we get when we merge a field with a setter?</p>
</li>
<li>
<p>Type of merge member: What is the return/parameter type of the merge result?</p>
</li>
<li>
<p>Accessibility: what is the accessibility of the merge result?</p>
</li>
<li>
<p>Optionality/Variadic: Should a parameter of the merge be optional or variadic?</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>The actual merge rules are implemented in the class <code>UnionMemberFactory</code> which delegates to either of the classes <code>UnionMethodFactory</code>, <code>UnionFieldFactory</code>, <code>UnionGetterFactory</code> and <code>UnionSetterFactory</code>.</p>
</div>
<div class="paragraph">
<p>In our example,
The merge result of <code>public TField m2: int</code> and <code>project TGetter m2(): int</code> are merged into a <code>project TGetter m2: int</code> .</p>
</div>
<div class="paragraph">
<p>Step 3: Construct the scope entry</p>
</div>
<div class="paragraph">
<p>In this final step, the actual IEObjectDescription for <code>m2</code> is constructed. In our example, since there exists one subscope exposing an <code>EObjectDescriptionWithError</code> (here <code>WrongWriteAccessDescription</code>), the final result is an instance of <code>UnionMemberDescriptionWithError</code>. This error instance is recognized during the linking phase and the error message of the subscope regarding <code>WrongWriteAccessDescription</code> is displayed: <em>Union combines fields and getters with name m2 and therefore property m2 is read-only.</em></p>
</div>
<div class="paragraph">
<p>More details can be found in the API documentation in the code. A good starting point is the class <code>ComposedMemberScope</code>.</p>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Binding_of_Structurally_References_Types"><a class="anchor" href="#sec:Binding_of_Structurally_References_Types"></a><a class="link" href="#sec:Binding_of_Structurally_References_Types">9.9. Structurally References Types</a></h3>
<div class="paragraph">
<p>Scoping of structurally referenced types is similar to binding of members. The structural typing modifier basically filters the members of a type. That is, the structural modifier filters out all non-public members, and the field-only modifier only accept fields. Thus, similar to accessibility aware scoping (<a href="#sec:Accessibility_of_types_and_members">Accessibility of types and members</a>), the <code>TypingStrategyAwareMemberScope</code> encapsulates an original scope and applies these additional filters.</p>
</div>
<div class="paragraph">
<p>Bindings to additional members of a structurally referenced type is implemented in <code>MemberScopeProvider.members(ParameterizedTypeRefStructural ..)</code>. Note that the current implementation does not necessarily bind to the type model (TModule) instance, as these members are part of a type reference. That is, usually these bindings refer to the <a href="#AC">AST</a> elements. Thus, it is not possible to compare these members directly, instead, a structural comparison has to be applied.</p>
</div>
</div>
<div class="sect2">
<h3 id="sec:Building"><a class="anchor" href="#sec:Building"></a><a class="link" href="#sec:Building">9.10. Building</a></h3>
<div class="sect3">
<h4 id="sec:Build_Phases"><a class="anchor" href="#sec:Build_Phases"></a><a class="link" href="#sec:Build_Phases">9.10.1. Build Phases</a></h4>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Phase 0
</td>
<td class="hdlist2">
<p>Loading Resources</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Phase 1: prelinking
</td>
<td class="hdlist2">
<p>Create symbols for all resources, includes creation of temporary pre-linked type models</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Phase 2: linking
</td>
<td class="hdlist2">
<p>Resolve all links, includes fully-resolved typed models<br>
includes compilation</p>
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>That is, not each resource is loaded, pre-linked and linked separately. Instead, all resources are first loaded, then all resources are pre -inked, and only then all resources are linked.</p>
</div>
</div>
<div class="sect3">
<h4 id="sec:Build_Scenarios"><a class="anchor" href="#sec:Build_Scenarios"></a><a class="link" href="#sec:Build_Scenarios">9.10.2. Build Scenarios</a></h4>
<div class="paragraph">
<p>Consequences:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>do not try to set any types in types builder, only create symbols there (probably not even members of types)</p>
</li>
</ul>
</div>
</div>
<div class="sect3">
<h4 id="sec:Lazy_linking_problem"><a class="anchor" href="#sec:Lazy_linking_problem"></a><a class="link" href="#sec:Lazy_linking_problem">9.10.3. Lazy linking problem</a></h4>
<div class="paragraph">
<p>Lazy linking proxies in the indes may trigger reloading of AST (which leads to invalid disconnected type models):</p>
</div>
<div class="paragraph">
<p>Lazy links (ending with |x in which x is an index entry of a temporary list used to resolve the link) must not be written into index.</p>
</div>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Proxies_and_Proxy_Resolution"><a class="anchor" href="#sec:Proxies_and_Proxy_Resolution"></a><a class="link" href="#sec:Proxies_and_Proxy_Resolution">9.11. Proxies and Proxy Resolution (Overview)</a></h3>
<div class="paragraph">
<p>Here we give a brief overview of the different kinds of proxies and when / how they are created and resolved.</p>
</div>
<div class="sect3">
<h4 id="xtexts-lazy-linking-proxies"><a class="anchor" href="#xtexts-lazy-linking-proxies"></a><a class="link" href="#xtexts-lazy-linking-proxies">9.11.1. Xtext’s Lazy Linking Proxies</a></h4>
<div class="ulist">
<ul>
<li>
<p>URI fragment is <code>|</code> <math xmlns="http://www.w3.org/1998/Math/MathML"><mi>n</mi></math> (where <math xmlns="http://www.w3.org/1998/Math/MathML"><mi>n</mi></math> is a non-negative integer).<br>
<code>platform:/resource/Project/src/A.n4js#|3</code></p>
</li>
<li>
<p>created by Xtext’s <code>LazyLinkingResource</code> in the AST after parsing (they are only ever created in the AST, but the types builder may copy them to the TModule, so they may appear there as well.</p>
</li>
<li>
<p>used to represent cross-references defined in the source code (i.e. name of an identifiable element used in source code to refer to that element).<br></p>
<div class="paragraph">
<p>Since the types builder sometimes copies proxies from AST to TModule (e.g. type of an element that was provided with an explicit type declaration in the source code), these proxies may also appear in the TModule, but only between the types builder phase and the end of the post-processing phase (or later, in case they are unresolvable).</p>
</div>
</li>
<li>
<p>resolution is handled by <code>#getEObject(String)</code> in <code>LazyLinkingResource</code>, which recognizes lazy linking URI fragments and then forwards them to <code>#getEObject(String,Triple)</code>, which in turn relies on the Xtext infrastructure.</p>
</li>
<li>
<p>latest time of resolution: post processing. After post processing has completed, they should all be gone (unless they are unresolvable, e.g. typo in source code).</p>
</li>
<li>
<p>fun facts:</p>
<div class="ulist">
<ul>
<li>
<p>the number after the pipe character is the index of a <code>Triple</code> stored in field <code>proxyInformation</code> in each <code>LazyLinkingResource</code>.</p>
</li>
<li>
<p>the resource given before the fragment (e.g. <code>A.n4js</code> in the above example) is not the resource the proxy is pointing to (i.e. the resource containing the target EObject), but the resource from where the link originates.</p>
</li>
</ul>
</div>
</li>
</ul>
</div>
</div>
<div class="sect3">
<h4 id="standard-emf-proxies"><a class="anchor" href="#standard-emf-proxies"></a><a class="link" href="#standard-emf-proxies">9.11.2. Standard EMF Proxies</a></h4>
<div class="ulist">
<ul>
<li>
<p>URI fragment contains a path to an EObject, using reference names and indices:<br>
<code>platform:/resource/Project/src/A.n4js#/1/@topLevelTypes.1/@ownedMembers.0</code></p>
</li>
<li>
<p>created automatically by EMF</p>
<div class="ulist">
<ul>
<li>
<p>during deserialization of a TModule <math xmlns="http://www.w3.org/1998/Math/MathML"><mi>M</mi></math> from the Xtext index for all references to a different TModule <math xmlns="http://www.w3.org/1998/Math/MathML"><msup><mi>M</mi><mi>'</mi></msup></math> (see <code>UserdataMapper</code>).</p>
</li>
<li>
<p>when unloading a resource.</p>
</li>
<li>
<p>&#8230;&#8203;</p>
</li>
</ul>
</div>
</li>
<li>
<p>used to represent</p>
<div class="ulist">
<ul>
<li>
<p>cross-references from one TModule to another TModule.</p>
</li>
<li>
<p><code>astElement</code> links from TModule to AST whenever the AST is not present (e.g. resource was loaded from Xtext index).</p>
</li>
<li>
<p><code>definedType</code> links from AST to TModule after deleting the TModule (this happens in the incremental builder after the pre-linking phase).</p>
</li>
<li>
<p>all kinds of links after demand-loading an AST by resolving an <code>astElement</code> link (pathological case).</p>
</li>
</ul>
</div>
</li>
<li>
<p>resolution is handled in two ways:</p>
<div class="ulist">
<ul>
<li>
<p>if the context <code>EObject</code> of the proxy, i.e. the one where the proxified cross-reference originates, is contained in an N4JSResource <math xmlns="http://www.w3.org/1998/Math/MathML"><mi>R</mi></math>, then resolution is handled by <code>N4JSResource#doResolveProxy()</code> (see also <code>ProxyResolvingResource</code> for details).</p>
<div class="paragraph">
<p>In this case, special handling is performed to make sure that (a) the target resource is loaded from the index, if possible, and (b) post-processing of the target resource is initiated iff the target resource was loaded from AST (instead from the Xtext index) AND post-processing of resource <math xmlns="http://www.w3.org/1998/Math/MathML"><mi>R</mi></math> is currently in progress or has already been completed.</p>
</div>
</li>
<li>
<p>otherwise, resolution is handled by standard EMF functionality.</p>
</li>
</ul>
</div>
</li>
<li>
<p>latest time of resolution: none. In fact, some of those proxies (those representing <code>astElement</code> links from TModule to AST) must not be resolved at all, because this is not yet properly handled.</p>
</li>
</ul>
</div>
</div>
<div class="sect3">
<h4 id="_how-is-proxy-resolution-triggered"><a class="anchor" href="#_how-is-proxy-resolution-triggered"></a><a class="link" href="#_how-is-proxy-resolution-triggered">9.11.3. How is Proxy Resolution Triggered?</a></h4>
<div class="paragraph">
<p>Resolution of proxies throughout the N4JS implementation is triggered as usually when using EMF, which means: whenever the getter of a EMF cross-reference is invoked and the value is still a proxy, the EMF-generated code of the getter will automatically trigger resolution of this proxy. For details look at the EMF-generated code of the getter of any cross-reference (<code>IdentifierRefImpl#getId()</code> would be a good example).</p>
</div>
</div>
<div class="sect3">
<h4 id="_when-is-proxy-resolution-allowed"><a class="anchor" href="#_when-is-proxy-resolution-allowed"></a><a class="link" href="#_when-is-proxy-resolution-allowed">9.11.4. When is Proxy Resolution Allowed?</a></h4>
<div class="paragraph">
<p>So, at what time is it legal to trigger such a proxy resolution? Or, more concretely, during which resource load states (<a href="#sec:N4JS_Resource_Load_States">N4JS Resource Load States</a>) is it legal to trigger proxy resolution? In fact, asking the question in this way is incorrect or at least not very helpful, because the answer would be (almost) always. The better question is: which components of the system / which parts of the code base are allowed to trigger proxy resolution?</p>
</div>
<div class="paragraph">
<p>For example, triggering resolution is disallowed in the <code>ASTStructureValidator</code> and <code>N4JSTypesBuilder</code>, but for the outside client code such as a JUnit test it is allowed to trigger proxy resolution as early as right after parsing. For an example of the latter see test <code>#testStateFullyProcessed_triggeredOnlyThroughProxyResolution()</code> in <code>N4JSResourceLoadStatesTest</code>.</p>
</div>
<div class="paragraph">
<p>In summary, we can state the rule that the <strong>internal N4JS implementation</strong> must not trigger any proxy resolution until installation of the derived state has completed, i.e. not before resource load state "Fully Initialized", but <strong>client code</strong> may trigger proxy resolution as early as right after parsing, i.e. already in resource load state "Loaded".</p>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_validation"><a class="anchor" href="#_validation"></a><a class="link" href="#_validation">10. Validation</a></h2>
<div class="sectionbody">
<div class="admonitionblock warning">
<table>
<tr>
<td class="icon">
<i class="fa icon-warning" title=""></i>
</td>
<td class="content">
This chapter may be outdated.
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>In this chapter the concept for the validation infrastructure of the N4JS IDE is described.</p>
</div>
<div class="paragraph">
<p>Requirements:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>like in old IDE centralize issue codes and messages</p>
</li>
<li>
<p>instead of the DLTK API, the Xtext Validation API should be used</p>
</li>
<li>
<p>there should be an unified approach / call to produce validation messages (as in old IDE with the call of reportProblem)</p>
</li>
</ul>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:validation_overview"><a class="anchor" href="#sec:validation_overview"></a><a class="link" href="#sec:validation_overview">10.1. Validation overview</a></h3>
<div class="ulist">
<ul>
<li>
<p>in Xtext in most cases validation should triggered after all Xtext resources are linked (so everything is resolved), so most validations are defined in N4JSValidator and there in composed validators</p>
</li>
<li>
<p>in Xtext there are resource diagnostics and validation diagnostics</p>
<div class="ulist">
<ul>
<li>
<p>resource diagnostics are produced for issues related to found syntax errors and linking errors</p>
</li>
<li>
<p>validation diagnostics are produced for issues found during semantic validation (model invariants)</p>
</li>
<li>
<p>Note, that you can only produce diagnostics only for the resource currenlty validated - e.g. it isn’t possible to create a marker for a duplicate issue in the other resource that contains the first element while producing the issue for the second - you have to create the diagnostic when validating the other resource</p>
</li>
</ul>
</div>
</li>
<li>
<p>validation message sources</p>
<div class="ulist">
<ul>
<li>
<p>parser, e.g. ANTLR parser: messages are contained and produced internally by the parser but can be adapted by a customized SyntaxErrorMessageProvider (see link section below)</p>
</li>
<li>
<p>lazy linking - N4JSLinker (extension of lazy linker): custom handling exceptions from value converters, that produces own issues as XtextLinkingDiagnostic (this is a resource diagnostic)</p>
</li>
<li>
<p>lazy linking - ASTStructureValidator: traverses AST and produces custom DiagnosticMessage (triggered during N4JSLinker.doLinkModel), creates XtextLinkingDiagnostic (this is a resource diagnostic). The ASTStructureValidator checks for things like allowed occurence of continue, break, return as well as allowed labels</p>
</li>
<li>
<p>linking - default: uses the ILinkingDiagnosticMessageProvider to create error messages for typical error cases (referenced not resolved, unique constraint violation, bounds contraints violation, etc.)</p>
</li>
<li>
<p>linking - ErrorAwareLinkingService: In N4JS we have introduced a custom IEObjectDescription AbstractDescriptionWithError that holds issue code and error message and ErrorAwareLinkingService as extension of DefaultLinkingService to produce for every AbstractDescriptionWithError a XtextLinkingDiagnostic (this is a resource diagnostic) - the usual Xtext behavior would be to produce a linking error with linking disabled, with our customization linking still works but the error message/marker is produced as well. Currently there are these custom AbstractDescriptionWithError implementations, that are produced during scoping</p>
<div class="ulist">
<ul>
<li>
<p>AmbiguousImportDescription: indicate an ambiguous wildcard import</p>
</li>
<li>
<p>PlainAccessOfAliasedImportDescription: indicate accessing type directly instead of using the alias it has been imported with</p>
</li>
<li>
<p>InvisibleMemberDescription: indicate accessing a member, that is not visible to the caller due to access modifier restrictions</p>
</li>
<li>
<p>InvisibleTypeOrVariableDescription: indicate accessing a type / variable or function, that is not visible to the caller due to access modifier restrictions</p>
</li>
<li>
<p>WrongFieldAccessorDescription: setter/getter access used in wrong context</p>
</li>
<li>
<p>WrongStaticAccessorDescription: static/non-static access used in wrong context</p>
</li>
</ul>
</div>
</li>
<li>
<p>N4JSTypeSystem:</p>
<div class="ulist">
<ul>
<li>
<p>error: cannot type given expression</p>
</li>
<li>
<p>error: element is not a subtype of another type</p>
</li>
<li>
<p>RuleFailedException is thrown in type system, then handled as result where out of the contained information the AbstractDeclarativeValidator API is used to create issues via calls to error(..) or warning(..) methods</p>
</li>
<li>
<p>RuleFailedExceptionWithoutStacktrace - sub class of RuleFailedException</p>
</li>
<li>
<p>N4JSTypeValidator.createError &#8594; delegates to AbstractDeclarativeValidator, or directly calls AbstractDeclarativeValidator.error</p>
</li>
<li>
<p>XsemanticsValidatorErrorGenerator (currently not used)</p>
</li>
</ul>
</div>
</li>
<li>
<p>N4JSValidation: composed validators, declarative checks, i.e. using annotation @Check at methods, issues mostly created by using combination of message, context object, EFeature and index passed to AbstractDeclarativeValidator methods like error(..), warning(..)</p>
<div class="ulist">
<ul>
<li>
<p>N4JSClassValidator - checks related to classes, like duplicate member check</p>
</li>
<li>
<p>N4JSFunctionValidator - checks for function expressions in expression statements and creates an error, but this validation is already done by the ASTStructureValidator</p>
</li>
<li>
<p>N4JSImportValidator - checks invalid, duplicated and invalid imports</p>
</li>
<li>
<p>N4JSInterfaceValidator - currently checks only for static members</p>
</li>
<li>
<p>N4JSRoleValidator - currently checks only for static members</p>
</li>
<li>
<p>N4JSStrictValidator - checks for function expressions in expression statements and creates an error, but this validation is already done by the ASTStructureValidator, thus this composite test is commented out</p>
</li>
<li>
<p>N4JSTypeValidator - this is type system, see above</p>
</li>
</ul>
</div>
</li>
</ul>
</div>
</li>
<li>
<p>Check types and check modes</p>
<div class="ulist">
<ul>
<li>
<p>Check types are used in <code>@Check</code> annotation to influence, when a validation is triggered</p>
<div class="ulist">
<ul>
<li>
<p><code>FAST</code> - Check is declared as fast</p>
</li>
<li>
<p><code>NORMAL</code> - the common checks</p>
</li>
<li>
<p><code>EXPENSIVE</code> - long running checks</p>
</li>
</ul>
</div>
</li>
<li>
<p>Check modes decide using check types when to run which types of checks</p>
<div class="ulist">
<ul>
<li>
<p><code>FAST_ONLY</code> - used by Xtext editor to execute FAST checks on every key stroke and in embedded Xtext editors</p>
</li>
<li>
<p><code>NORMAL_ONLY</code> - not used in Xtext framework itself</p>
</li>
<li>
<p><code>EXPENSIVE_ONLY</code> - not used in Xtext framework itself</p>
</li>
<li>
<p><code>NORMAL_AND_FAST</code> - used by Xtext editor to execute FAST and NORMAL checks on file save as well as on marker update for changed resources (delta calculated by Xtext builder)</p>
</li>
<li>
<p>ALL - executes all types of checks when invoking explicitley the Validate action in the context menu of the Xtext editor</p>
</li>
</ul>
</div>
</li>
<li>
<p>So by default check types and modes aren’t configurable at runtime</p>
</li>
</ul>
</div>
</li>
<li>
<p>Severity</p>
<div class="ulist">
<ul>
<li>
<p>types</p>
<div class="ulist">
<ul>
<li>
<p><code>ERROR</code> - indicates a serious issue, that later will prevent things like builder, compiler and so on, to run</p>
</li>
<li>
<p><code>WARNING</code> - indicates a possible semantic problem, where the user have to decide how to handle it. Issues with such severity won’t stop any post processing</p>
</li>
<li>
<p><code>INFO</code> - only an information hint for the user. Note, it is not allowed to create a diagnostic with severity INFO.</p>
</li>
</ul>
</div>
</li>
<li>
<p>can be statically provided by calling the AbstractDeclarativeValidator methods error(..), warning(..), info(..) or directly pass the severity to a sub class of AbstractValidationDiagnostic (e.g. FeatureBasedDiagnostic, RangeBasedDiagnostic)</p>
</li>
<li>
<p>can also determined dynamically at runtime with using the IssueSeveritiesProvider and a implementation of IPreferenceValuesProvider (e.g. the EclipsePreferencesProvider that uses the Eclipse preference store and preference page)</p>
</li>
</ul>
</div>
</li>
<li>
<p>Issue codes</p>
<div class="ulist">
<ul>
<li>
<p>used to identify an issue elsewhere, e.g. when applying an quickfix for a validation issue but also for configuring validation handling (e.g. in a Eclipse preference page).</p>
</li>
<li>
<p>We can use this issue code to also externalize the issue messages at a central place</p>
</li>
</ul>
</div>
</li>
<li>
<p>Issue data: String array to store additional data to be used in other places (e.g. hints for quickfixes)</p>
</li>
<li>
<p>message: The message shown as text for the marker created at the resource in Eclipse and shown in the Xtext editor but also available by the methods getWarnings and getErrors at the XtextResource itself and so usable when logging messages to console in headless mode</p>
</li>
</ul>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:validation_control_flow"><a class="anchor" href="#sec:validation_control_flow"></a><a class="link" href="#sec:validation_control_flow">10.2. Validation control flow</a></h3>
<div class="paragraph">
<p><a href="#fig:cd_validation">Validation control flow</a> gives an overview over the common control flow that triggers validation.</p>
</div>
<div id="fig:cd_validation" class="imageblock center">
<div class="content">
<img src="chapters/10_validation/images/cd_validation.svg" alt="cd validation">
</div>
<div class="title">Figure 24. Validation control flow</div>
</div>
<div class="paragraph">
<p>Validation is either triggered by dirty state handling (editing an Xtext document without saving starts a validation job) or by the automated build (invoked directly or started by resource changes in the project e.g. after saving a file).</p>
</div>
<div class="paragraph">
<p>While in dirty state handling the current resource is already parsed and resolved the builder have to load the resource.</p>
</div>
<div class="paragraph">
<p>All issues collected during load (i.e. during parsing, linking and scoping) the resource are added to the resource.</p>
</div>
<div class="paragraph">
<p>In the automated build process there is step <code>updateMarkers</code> that triggers the validation.</p>
</div>
<div class="paragraph">
<p>The <code>SourceContainerAwareResourceValidator</code> is a customization by us to handle only files that are contained in folders declared as source container by the package.json file.</p>
</div>
<div class="paragraph">
<p>The <code>CancelableDiagnostican</code>, called by the resource validator, iterates over all elements contained in the resource. For each element the bound validator is called, in our case <code>N4JSValidator</code>, as it is registered as validator for the N4JS EPackage (in <code>AbstractN4JSValidator</code>).</p>
</div>
<div class="paragraph">
<p>As this validator extends <code>AbstractDeclarativeValidator</code> in the first step all methods that are annotated with @Check and that have exactly one parameter are collected keyed by the type of their input parameter. The result of this collection process is cached. There is a defined order how the methods are collected:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>all local methods</p>
</li>
<li>
<p>all methods recursively found in the super classes of the current class</p>
</li>
<li>
<p>all methods found for the in the composed check annotation defined validators (by applying this algorithm as well)</p>
</li>
<li>
<p>all methods recursively found in the composed checks in the super classes of the current class (by applying this algorithm as well)</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>The <code>N4JSValidator</code> filters all methods that uses the type of the currently traversed element from the before collected check methods and invokes them with the element from the resource.</p>
</div>
<div class="paragraph">
<p>The Xsemantics type system validator is used as one of the composed validators in <code>N4JSValidator</code>. So although <code>N4JSValidator</code> extends <code>N4JSTypeSystemValidator</code>, <code>N4JSTypeSystemValidator</code> just re-uses the validation infrastructure but not its call hierarchy.</p>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:validation_issue_ids"><a class="anchor" href="#sec:validation_issue_ids"></a><a class="link" href="#sec:validation_issue_ids">10.3. Issue IDs and Messages</a></h3>
<div class="paragraph">
<p>For now the NLS validation message bundle resides in<br>
<code>/org.eclipse.n4js/src/org/eclipse/n4js/validation/messages.properties</code><br>
The entries in the messages.properties follows the pattern as described in <code>NLSProcessor</code>, the NLS class is <code>IssueCodes</code></p>
</div>
<div class="admonitionblock tip">
<table>
<tr>
<td class="icon">
<i class="fa icon-tip" title=""></i>
</td>
<td class="content">
We use the same pattern for semver and json.
</td>
</tr>
</table>
</div>
<div class="ulist">
<ul>
<li>
<p>IDs shouldn’t be to long, as there might be a lot of markers and the issue codes are stored in memory</p>
</li>
<li>
<p>the ID should encode where the issue has been created, therefore there should be common used prefixes like</p>
<div class="ulist">
<ul>
<li>
<p>PRS for parser (not used yet)</p>
</li>
<li>
<p>VCO for value converter</p>
</li>
<li>
<p>AST for issues found during AST traversal</p>
</li>
<li>
<p>LIN for issues found during scoping/linking (not used yet)</p>
</li>
<li>
<p>TYS for type system</p>
</li>
<li>
<p>VAL for semantic validation (not used yet)</p>
</li>
</ul>
</div>
</li>
<li>
<p>besides the source also the domain of the issue should be encoded (the following list may reduced or extended over time, when it gets obvious which sorts of domain specific validations are required in which frequency)</p>
<div class="ulist">
<ul>
<li>
<p>CLF for issues common to all classifiers</p>
</li>
<li>
<p>CLA for classes (not used yet)</p>
</li>
<li>
<p>ROL for roles</p>
</li>
<li>
<p>FUN for function</p>
</li>
<li>
<p>IMP for imports</p>
</li>
<li>
<p>VAR for variables (not used yet)</p>
</li>
<li>
<p>MEM for classifier members in general</p>
</li>
<li>
<p>OLI for object literals (not used yet)</p>
</li>
<li>
<p>ENU for enumerations (not used yet)</p>
</li>
<li>
<p>ARR for array literals (not used yet)</p>
</li>
<li>
<p>ANN for annotation related issues (not used yet)</p>
</li>
<li>
<p>EXP for expression related issues (assignment expression, binary expression, etc.) (not used yet)</p>
</li>
<li>
<p>STMT for statement related issues, such as if-else (conditional) , loops, switch etc.</p>
</li>
<li>
<p>PRP for property access related issues (not used yet)</p>
</li>
<li>
<p>EXC for exception handling related issues (not used yet)</p>
</li>
<li>
<p>LBL for labels related issues (not used yet)</p>
</li>
</ul>
</div>
</li>
<li>
<p>also technical validation aspects can be encoded</p>
<div class="ulist">
<ul>
<li>
<p>DUP for duplicate checks</p>
</li>
<li>
<p>VIS for visibility checks (public, private, export, etc.)</p>
</li>
<li>
<p>STR for issues related only applied in strict mode</p>
</li>
</ul>
</div>
</li>
<li>
<p>examples</p>
<div class="ulist">
<ul>
<li>
<p><code>IMP_AMBIGUOUS</code></p>
</li>
<li>
<p><code>VIS_ILLEGAL_MEMBER_ACCESS</code></p>
</li>
<li>
<p><code>CLF_ABSTRACT_FINAL</code></p>
</li>
<li>
<p><code>AST_RESERVED_IDENTIFIER</code></p>
</li>
<li>
<p><code>VCO_HEXINT_CONVERT_EMPTY_STR</code></p>
</li>
<li>
<p><code>TYS_NO_SUBTYPE</code></p>
</li>
</ul>
</div>
</li>
</ul>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:validation_usage_patterns"><a class="anchor" href="#sec:validation_usage_patterns"></a><a class="link" href="#sec:validation_usage_patterns">10.4. Usage Pattern</a></h3>
<div class="paragraph">
<p>Due to the different places and circumstances a real unification of the API wasn’t possible yet (and wouldn’t have made sense), so there are these different usage patterns</p>
</div>
<div class="ulist">
<ul>
<li>
<p>in a custom error aware EObjectDescription like WrongFieldAccessorDescription you just return the issue code in getIssueCode and the message created using the issue code as well as the replacements for the wildcards in getMessage</p>
</li>
<li>
<p>in a validator extending AbstractDeclarativeValidator you just call <code>addIssue(message, context, EFeature, issueCode)</code>. The message you have to create before by calling the corresponding <code>getMessageFor[ISSUE_ID]</code> method passing the required wildcard replacement</p>
</li>
<li>
<p>in the ASTStructureValidator you have to call <code>producer.addDiagnostic(new DiagnosticMessage(IssueCodes.messageFor[ISSUE_ID](wildcard1, wildcard2, ..), IssueCodes.getDefaultSeverity(IssueCodes.[ISSUE_ID]), IssueCodes.[ISSUE_ID]))</code></p>
</li>
<li>
<p>in the custom value converters you have to pass the information to an exception, so the call is: <code>new N4JSValueConverterException(IssueCodes.getMessageFor[ISSUE_ID](wildcard1, wildcard2, ..), IssueCodes.[ISSUE_ID], node, null)</code>. Beside this exception also N4JSValueConverterWithValueException is used in some places. In N4JSLinker then these exceptions are catched and a DiagnosticMessage is created out of the informations contained in these exceptions.</p>
</li>
<li>
<p>As Xsemantics uses hard wired error or warning in its grammar you cannot adapt these places, but there are currently only three messages produced by Xsemantic (cannot type, not a sub type, null object passed to system). They are all handled in N4JSTypeValidator.createError where the message from Xsemantic is split up in its parts and then passed as wild card replacements to e.g. <code>IssueCodes.getMessageForTYS_NO_SUBTYPE</code>.</p>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="sec:validation_links"><a class="anchor" href="#sec:validation_links"></a><a class="link" href="#sec:validation_links">10.5. Links</a></h3>
<div class="ulist">
<ul>
<li>
<p><a href="http://www.eclipse.org/Xtext/documentation.html#validation">Xtext Runtime Concepts: Validation</a></p>
</li>
<li>
<p><a href="http://blog.dietmar-stoll.de/2013/04/multiple-validators-in-xtext.html">Multiple validators in Xtext</a></p>
</li>
<li>
<p>Customize error messages</p>
<div class="ulist">
<ul>
<li>
<p><a href="http://zarnekow.blogspot.de/2010/06/customizing-error-messages-in-xtext-10.html">Customizing error messages in Xtext</a></p>
</li>
<li>
<p><a href="http://blog.dietmar-stoll.de/2012/07/custom-syntax-error-messages-with-quick.html">Custom syntax error messages with quickfix</a></p>
</li>
<li>
<p><a href="http://stackoverflow.com/questions/14526524/xtext-customizing-error-msg-by-unordered-groups">Xtext: customizing error messages by unordered groups</a></p>
</li>
</ul>
</div>
</li>
</ul>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_references"><a class="anchor" href="#_references"></a><a class="link" href="#_references">11. References</a></h2>
<div class="sectionbody">
<div class="admonitionblock warning">
<table>
<tr>
<td class="icon">
<i class="fa icon-warning" title=""></i>
</td>
<td class="content">
This chapter maybe outdated.
</td>
</tr>
</table>
</div>
<div class="sect2">
<h3 id="sec:usecases"><a class="anchor" href="#sec:usecases"></a><a class="link" href="#sec:usecases">11.1. Use cases</a></h3>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Compilation
</td>
<td class="hdlist2">
<p>for deciding in incremental builder which resources requires a recompilation</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Editing
</td>
<td class="hdlist2">
<p>Dirty state calculation: for deciding which resources needs to be reparsed as references have changed</p>
</td>
</tr>
<tr>
<td class="hdlist1">
UI
</td>
<td class="hdlist2">
<p>Such as <kbd>Find references</kbd>, find all places in the workspaces that points to the selected element</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Tools
</td>
<td class="hdlist2">
<p>requiring references, such as refactorings, e.g., rename refactoring: apply the renaming of the element also to all references to it (like find by references)</p>
</td>
</tr>
</table>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:calculation_algorithm"><a class="anchor" href="#sec:calculation_algorithm"></a><a class="link" href="#sec:calculation_algorithm">11.2. Calculation algorithm</a></h3>
<div class="sect3">
<h4 id="sec:Xtext_default_implementation"><a class="anchor" href="#sec:Xtext_default_implementation"></a><a class="link" href="#sec:Xtext_default_implementation">11.2.1. Xtext default implementation</a></h4>
<div class="paragraph">
<p>Using Reference Descriptions:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>default implementation contained in method <code>createReferenceDescriptions</code> of <code>o.e.x.resource.impl.DefaultResourceDescriptionStrategy</code></p>
</li>
<li>
<p>iterates over all EReferences of the EClass of the current element</p>
</li>
<li>
<p>navigates all references and resolves them (already done before in DefaultResourceDescription.computeReferenceDescriptions)</p>
</li>
<li>
<p>create reference description objects for all these references</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>In case of N4JS and the types, reference descriptions are also created for references to types model elements (definedType) and for references from Types element to AST.</p>
</div>
<div class="paragraph">
<p>We do not use this default implementation for two reasons:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>expensive</p>
</li>
<li>
<p>Default implementation of reference descriptions only calculates the direct dependencies but not the transitive ones + the calculation of the URIs is very expensive.</p>
</li>
</ol>
</div>
</div>
<div class="sect3">
<h4 id="sec:N4_implementation"><a class="anchor" href="#sec:N4_implementation"></a><a class="link" href="#sec:N4_implementation">11.2.2. N4JS implementation</a></h4>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>the Xtext default implementation is disabled by let <code>N4JSResourceDescription.computeReferenceDescriptions</code> return an empty list. Also the bound <code>N4JSDescriptionUtils</code> returns an empty list for collectOutgoingReferences</p>
</li>
<li>
<p>Calculating <em>direct references</em> are only done inside <code>N4JSResourceDescription.getImportedNames</code> (that uses newly introduced <code>N4JSCrossReferenceComputer.computeCrossRefs</code> for collecting all direct dependencies) - here only (parameterized) type refs, types and identifiable elements are collected</p>
</li>
<li>
<p>collect all <em>transitive dependencies</em>, i.e. all super classes, consumed roles and implemented interfaces in the type hierarchy and add their resources as dependency (this is done in <code>N4JSResourceDescription.getImportedNames</code> (after calculating all direct dependencies with <code>N4JSResourceDescriptionStrategy</code>)</p>
</li>
<li>
<p>bind <code>N4JSReferenceQueryExecutor</code> as a custom implementation to calculate the target URIs for an given target element and bind <code>N4JSReferenceFinder</code> as a custom implementation to calculate reference descriptions to be used by the default Xtext found references UI (use case UI)</p>
</li>
</ol>
</div>
<div class="paragraph">
<p><code>ClusteringBuilderState.doUpdate</code> calculates if a dependent resource has changed (in the context of calculating the <code>DefaultResourceDescriptionDelta</code> out of the old and new resource descriptions). Each resource description consists of <code>EObjectDescriptions</code>. The <code>EObjectDescription</code> for <code>Script</code> also contains the types model (<code>TModule</code>) for the resource. The references between resources are implemented via the type model. If it has changed (compared with the user data of the old <code>EObjectDescription</code>) then all other resource descriptions registered as been dependent on the resource (the qualified names of the resource descriptions are serialized as imported names within the resource description) will be notified that a reparse is needed.</p>
</div>
<div class="paragraph">
<p>For <em>dirty state</em> the same behavior is achieved via the dirty state editor support using the resource set of the editor (instead the file system resources).</p>
</div>
<div class="paragraph">
<p>As the primitive and built-in types are fixed they are ignored when calculating the dirty state. When calculating dependending resources and dirty state the reference of an AST element to its defining type is ignored as is the reference from the type to its AST element</p>
</div>
<div class="paragraph">
<p><a href="#fig:cd_classes">Classes</a> shows the different entry points (user actions) and classes involved in the process.</p>
</div>
<div id="fig:cd_classes" class="imageblock center">
<div class="content">
<img src="chapters/11_references/images/cd_classes.svg" alt="title-"Reference handling"">
</div>
</div>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:PerformanceOfDependencyCalculation"><a class="anchor" href="#sec:PerformanceOfDependencyCalculation"></a><a class="link" href="#sec:PerformanceOfDependencyCalculation">11.3. Performance Of Dependency Calculation</a></h3>
<div class="paragraph">
<p>Concerning frequency and contexts it is clear, that triggering <code>Find references</code> and rename refactoring is not as frequent as editing (n4)js files that leads to dirty state (very often as happens when editing) and to trigger compilation (at file save, also often). Calculating if a resource is affected by a change (dirty state calculation) may not be too expensive. But running compilation for too many files (or the wrong set of files) due to incorrect dirty state calculation is expensive.</p>
</div>
<div class="paragraph">
<p><code>N4JSResourceDescription.getImportedNames</code> is invoked on every edit of a file in the editor, so on every edit the complete content has to be retraversed for possible new references (expensive but not avoidable). For the types of all found references the super types have to recalculated. Traversing the type hierarchy shouldn’t be too expensive.</p>
</div>
<div class="paragraph">
<p>Possible optimization could be:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>caching of traversing referenced types whose resources had not changed since last edit</p>
</li>
<li>
<p>not traversing types that are imported but non of their features are used within the current resource</p>
</li>
</ol>
</div>
<div class="paragraph">
<p><em>Those optimization should be done only if there are real performance issues are discovered.</em></p>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:kinds_of_references"><a class="anchor" href="#sec:kinds_of_references"></a><a class="link" href="#sec:kinds_of_references">11.4. Kinds of references</a></h3>
<div class="sect3">
<h4 id="sec:Cross_References_to_be_ignored"><a class="anchor" href="#sec:Cross_References_to_be_ignored"></a><a class="link" href="#sec:Cross_References_to_be_ignored">11.4.1. Cross References to be ignored</a></h4>
<table class="tableblock frame-all grid-all spread">
<colgroup>
<col style="width: 33.3333%;">
<col style="width: 33.3333%;">
<col style="width: 33.3334%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Element</th>
<th class="tableblock halign-left valign-top">Reference</th>
<th class="tableblock halign-left valign-top">Explanation</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">TypeDefiningElement</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">definedType: Type</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">always inner resource change (e.g., Functions, Classifier)</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">types::Type</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">astElement</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">as always inner resource change</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">ImportDeclaration</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">importedModule: TModule</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">only affected if the resource name changes (and such a change cannot be performed dirty)</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">ContinueStatement</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">label: LabelledStatement</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">always inner resource changes</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">BreakStatement</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">label: LabelledStatement</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">always inner resource changes</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">types::PrimitiveType</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">autocoercedObject: TClassifier</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">fixed (immutable) and internal</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect3 language-n4js">
<h4 id="sec:Cross_References_to_be_handled"><a class="anchor" href="#sec:Cross_References_to_be_handled"></a><a class="link" href="#sec:Cross_References_to_be_handled">11.4.2. Cross References to be handled</a></h4>
<div class="paragraph">
<p>Cross References</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>types::ParameterizedTypeRef &#8594; declaredType : Type</p>
</li>
<li>
<p>* &#8594; N4GetterDeclaration: N4FieldDeclaration type, but references to getter can be done also done from outside</p>
</li>
<li>
<p>* &#8594; N4SetterDeclaration: undef, references to setter can be done also done from outside</p>
</li>
<li>
<p>types::PrototypeTypeRef &#8594; type : Type</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>Contained <code>ParameterizedTypeRef</code> and <code>TypeVariables</code> (that internally references to <code>Type</code>):</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>references to declared super types of a type (Class, Role, Interface), i.e. superType, consumedRoles, implementedInterfaces</p>
</li>
<li>
<p>TypeVariable &#8594; declaredUpperBounds</p>
</li>
<li>
<p>References in type arguments:</p>
<div class="ulist">
<ul>
<li>
<p>Wildcards &#8594; upper and lower bounds, e.g. <code>var List&lt;? super A&gt; l1;</code></p>
</li>
<li>
<p>direct type references, e.g. <code>var List&lt;A&gt; l;</code></p>
</li>
</ul>
</div>
</li>
</ol>
</div>
<div class="paragraph">
<p>Cross References to IdentifiableElement (types):</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>IdentifierRef &#8594; id : IdentifiableElement</p>
</li>
<li>
<p>NamedImportSpecifier &#8594; importedElement : IdentifiableElement</p>
</li>
<li>
<p>ParameterizedPropertyAccessExpression &#8594; property : IdentifiableElement</p>
</li>
<li>
<p>PropertyAccessExpression &#8594; property : IdentifiableElement</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>Contained IdentifierRef (that internally references to IdentifiableElement):</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>ParameterizedCallExpression &#8594; target</p>
</li>
<li>
<p>as PrimaryExpression in MemberExpression</p>
</li>
</ol>
</div>
</div>
</div>
<div class="sect2">
<h3 id="sec:transitive_dependencies"><a class="anchor" href="#sec:transitive_dependencies"></a><a class="link" href="#sec:transitive_dependencies">11.5. Transitive dependencies</a></h3>
<div class="paragraph">
<p>Besides the direct dependencies we also need the transitive dependencies, as demonstrated in the following example.</p>
</div>
<div id="ex:transdepex" class="listingblock">
<div class="title">Transitive Dependency</div>
<div class="content">
<pre class="highlight"><code class="language-java" data-lang="java">export class A {
public myMethod()
}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="title">Transitive Dependency pt.2</div>
<div class="content">
<pre class="highlight"><code class="language-java" data-lang="java">export class B extends my/test/A {
}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="title">Transitive Dependency pt.3</div>
<div class="content">
<pre class="highlight"><code class="language-java" data-lang="java">export class C extends my/test/B {
myMethodC() {
this.myMethod()
}
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>If the name of <code>myMethod</code> in A changes, C should get dirty. This can get more complicated, if, e.g., a method in a consumed role is renamed, which then leads to binding references to super types.</p>
</div>
<div class="paragraph">
<p>Therefore all direct and indirect super types are calculated (including super classes, consumed roles and implemented interfaces) for all found directly referenced types. The qualified names of their resources are added to the list of imported names. <sup class="footnote">[<a id="_footnoteref_12" class="footnote" href="#_footnote_12" title="View footnote.">12</a>]</sup></p>
</div>
<div class="paragraph">
<p>Other transitive dependencies:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>call of member mixed by a consumed role</p>
<div class="olist loweralpha">
<ol class="loweralpha" type="a">
<li>
<p>role is consumed by role consumed by this class</p>
</li>
<li>
<p>role is consumed by class inherited by this class</p>
</li>
</ol>
</div>
</li>
<li>
<p>call of member available by implemented interface</p>
<div class="olist loweralpha">
<ol class="loweralpha" type="a">
<li>
<p>interface is implemented by role consumed by this class</p>
</li>
<li>
<p>interface is implemented by class inherited by this class</p>
</li>
</ol>
</div>
</li>
<li>
<p>call of member available by extended class</p>
<div class="olist loweralpha">
<ol class="loweralpha" type="a">
<li>
<p>class is extended by class inherited by this class</p>
</li>
</ol>
</div>
</li>
<li>
<p>chained method calls</p>
<div class="olist loweralpha">
<ol class="loweralpha" type="a">
<li>
<p>method is of type that itself has members which are directly called, so the type is not directly imported or referenced by name in the caller but indirectly required</p>
</li>
<li>
<p>method is of type that itself inherits members which are directly called, so the type (and its super types) is not directly imported or referenced by name in the caller but indirectly required</p>
</li>
</ol>
</div>
</li>
</ol>
</div>
<div class="exampleblock">
<div class="content">
<div class="paragraph">
<p>Each type is defined in its own file.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-java" data-lang="java">export class MyClassOne {
myMethodOne() {
var MyClassTwo instance;
instance.getElement().myMethodThree()
}
}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-java" data-lang="java">export class MyClassTwo {
MyClassThree getElement() {
return new MyClassThree;
}
}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-java" data-lang="java">export class MyClassThree {
void myMethodThree() {}
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>If <code>myMethodThree</code> is renamed this should affect MyClassOne.</p>
</div>
<div class="paragraph">
<p>Note that the method call in <code>MyClassOne</code> directly binds to the method in <code>MyClassThree</code>. However, the dependencies are only managed by means of types. So, from that perspective, the dependency between <code>MyClassOne</code> and <code>MyClassThree</code> is indirect.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-java" data-lang="java">export class MyClassOne {
void myMethodOne() {
var MyClassTwo instance;
instance.myMethodTwo().getElement().myMethodFour()
}
}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-java" data-lang="java">export class MyClassTwo {
MyClassThree&lt;MyClassFour&gt; myMethodTwo() {
return null;
}
}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-java" data-lang="java">export class MyClassThree&lt;T extends MyClassFour&gt; {
T element;
T getElement() {
return this.element;
}
}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-java" data-lang="java">export class MyClassFour {
void myMethodFour() {
}
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>If <code>myMethodFour</code> is renamed this should affect <code>MyClassOne</code>.</p>
</div>
</div>
</div>
<div class="paragraph">
<p>More examples are found in the tests (cf. <code>..ide.n4js.dirtystate.BuilderParticipantPluginTest</code> and <code>&#8230;&#8203;BuilderParticipantPluginUITest</code>)</p>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:find-references"><a class="anchor" href="#sec:find-references"></a><a class="link" href="#sec:find-references">11.6. Find references</a></h3>
<div class="paragraph">
<p>Find references is perceived as a feature in Eclipse IDE, but its implementation can also be useful in a headless scenario, e.g. in the compiler to drop dead code.
Therefore, as opposed to the Xtext default implementations, the code was refactored to split the parts that depend on the UI from the non-UI dependent logic (see <code>org.eclipse.n4js.findReferences</code> vs. <code>org.eclipse.n4js.ui.search</code>).</p>
</div>
<div class="sect3">
<h4 id="_background"><a class="anchor" href="#_background"></a><a class="link" href="#_background">11.6.1. Background</a></h4>
<div class="paragraph">
<p>Since no reference descriptions are stored in the index for N4JS resources, the cross references have to be found by other means.
That is, the list of imported names is used as an indicator to find resources that have a potential dependency to the searched element.
These resources have to be checked thoroughly.
That is, their clear text representation is checked at a first step against the clear text representation of the found element before the resource is fully loaded and cross references are resolved.</p>
</div>
<div class="paragraph">
<p>The decision to drop reference descriptions from the index was deliberate since they would only report bogus information in the context of inheritance, e.g. a method <code>getA</code> of type <code>B</code> my be overridden by <code>getA</code> in type <code>C</code>.
Concrete bindings against <code>C.getA</code> should also be reported as references to <code>B.getA</code> since they identify the same public API of the type hiearchy around <code>B</code>.
Therefore reference descriptions could not be used to find dependencies between source snippets.</p>
</div>
</div>
<div class="sect3">
<h4 id="_how-find-references-work"><a class="anchor" href="#_how-find-references-work"></a><a class="link" href="#_how-find-references-work">11.6.2. How Find References Work</a></h4>
<div class="paragraph">
<p>Methods for finding references are provided Xtext&#8217;s interface <code>IReferenceFinder</code> and can be used both by the UI or headlessly.
The N4JS implementation of this interface for the N4JS language is the class <code>ConcreteSyntaxAwareReferenceFinder</code>.
One of the key methods defined by the <code>IReferenceFinder</code> is <code>void findAllReferences(TargetURIs, IResourceAccess, IResourceDescriptions, Acceptor, IProgressMonitor)</code> that finds all places in all resources of the index whereby those places cross-reference one of the URIs contained in <code>TargetURIs</code> .</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>TargetURIs</code> contains the set of URIs to be searched. The caller of <code>IReferenceFinder</code> is responsible for collecting the <code>Target URIs</code> to be searched.</p>
</li>
<li>
<p><code>IResourceAccess</code> is used to search local references. This is needed because local references are usually not index.</p>
</li>
<li>
<p><code>IResourceDescriptions</code> is the indexed.</p>
</li>
<li>
<p><code>Acceptor</code> is called when a reference is found.</p>
</li>
<li>
<p><code>IProgressMonitor</code> is used for showing progress bar (can be null).</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>In the following, we will have a look at the workflow to find references when triggered in the UI.
After understanding the UI case, the workflow of find references in the headless case should be self-explanatory.</p>
</div>
<div class="paragraph">
<p><a href="#fig:findreference_workflow">Find reference workflow</a> shows the workflow of find references when triggered in the UI.</p>
</div>
<div id="fig:findreference_workflow" class="imageblock center">
<div class="content">
<img src="chapters/11_references/images/findreference_workflow.svg" alt="title-"Find reference workflow"">
</div>
</div>
<div class="paragraph">
<p>The following example will be used for explanation.</p>
</div>
<div class="listingblock">
<div class="title">A.n4js</div>
<div class="content">
<pre class="highlight"><code class="language-n4js" data-lang="n4js">import {B} from "B";
let b = new B(); // B here is an IdentifierRef referring to TClass B in B.n4js</code></pre>
</div>
</div>
<div class="listingblock">
<div class="title">B.n4js</div>
<div class="content">
<pre class="highlight"><code class="language-n4js" data-lang="n4js">export public class B {}</code></pre>
</div>
</div>
<div class="sect4">
<h5 id="_step-1-convert-cursor-position-to-declared-element"><a class="anchor" href="#_step-1-convert-cursor-position-to-declared-element"></a><a class="link" href="#_step-1-convert-cursor-position-to-declared-element">11.6.2.1. Step 1: Convert Cursor Position to Declared Element</a></h5>
<div class="paragraph">
<p>This step is represented by the purple color in <a href="#fig:findreference_workflow">Find reference workflow</a> diagram.</p>
</div>
<div class="paragraph">
<p>In the IDE, for the sake of convenience, we allow the user to find references of an arbitrary element at the current cursor.
For instance, while the cursor is currently at <code>IdentifierRef B</code> in the <code>NewExpression</code> in <code>A.n4js</code>, the user may want to find all references to <code>B</code>.
In those cases, we first need to find declaration element of <code>IdentifierRef B</code> which is <code>TClass B</code>. The Target URIs then contains a single URI to <code>TClass B</code>.
In diagram <a href="#fig:findreference_workflow">Find reference workflow</a>, the classe <code>EObjectAtOffsetHelper</code> can convert the current cursor position into a declared element.</p>
</div>
</div>
<div class="sect4">
<h5 id="_step-2-convert-declared-element-to-target-uris"><a class="anchor" href="#_step-2-convert-declared-element-to-target-uris"></a><a class="link" href="#_step-2-convert-declared-element-to-target-uris">11.6.2.2. Step 2: Convert Declared Element to Target URIs</a></h5>
<div class="paragraph">
<p>This step is represented by the yellow color in <a href="#fig:findreference_workflow">Find reference workflow</a> diagram.</p>
</div>
<div class="paragraph">
<p>The Target URIs contains the URIs whose references are to be searched.
The caller guarantees that <em>Target URIs contain only URIs to declared elements, i.e. definitions</em>.
For example, if we want to find references for <code>N4ClassDeclaration B</code> in <code>B.n4js</code>, the target URIs contains a URI to the AST node <code>N4ClassDeclaration B</code> and a URI to the TModule node <code>TClass B</code>.
Note that, in addition to the URI to the AST node <code>N4ClassDeclaration B</code>, the URI to the derived TModule node <code>TClass B</code> is also needed because <code>N4ClassDeclaration</code> can never be a target of a cross reference.
In the diagram <a href="#fig:findreference_workflow">Find reference workflow</a> , the classes depicted in yellow color are responsible for converting declared elements to <code>Target URIs</code> taking care of the derived <code>TModule</code> nodes.</p>
</div>
</div>
<div class="sect4">
<h5 id="_step-3-filter-potential-resources"><a class="anchor" href="#_step-3-filter-potential-resources"></a><a class="link" href="#_step-3-filter-potential-resources">11.6.2.3. Step 3: Filter Potential Resources</a></h5>
<div class="paragraph">
<p>This step is represented by the green color in <a href="#fig:findreference_workflow">Find reference workflow</a> diagram.</p>
</div>
<div class="paragraph">
<p>The general algorithm for finding references is to traverse the AST of each resource in the index and check each AST node if it has a cross reference to one of the URI in the <code>Target URIs</code>.
However, this is too expensive because potentially all resources in the index have to be loaded.
We need some way to quickly decide for a resource description if the corresponding resource may potentially contain the references before actually loading it for a more thorough search.
This is done using two pieces of information:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>typesOrModulesToFind</code>: the set containing the fully qualified names of the <code>type</code> and <code>module</code> of the declaration to be searched. This set is calculated in the class <code>TargetURIKey</code>.</p>
</li>
<li>
<p><code>imported names</code>: the set exposed by <code>ResourceDescription</code> that contains the types needed by the underlying resource. The implementation for calculating imported names can be found in the class <code>N4JSResourceDescription</code>.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>In our example, supposed that we are finding references for <code>class B</code>. The <code>typesOrModulesToFind</code> contains fully qualified names to <code>N4ClassDeclaration B</code> and <code>module B</code>, i.e. <code>B.B</code> and <code>B</code>.
The <code>imported names</code> of the resource description of <code>A.n4js</code> contains fully qualified names to <code>module B</code>, <code>class B</code>, i.e. <code>B</code> and <code>B.B</code>.
Since the set of imported names of <code>A.n4js</code> contains elements in <code>typesOrModulesToFind</code>, this resource is searched thoroughly for references.</p>
</div>
</div>
<div class="sect4">
<h5 id="_step-4-search-references-in-resource"><a class="anchor" href="#_step-4-search-references-in-resource"></a><a class="link" href="#_step-4-search-references-in-resource">11.6.2.4. Step 4: Search References in Resource</a></h5>
<div class="paragraph">
<p>If a resource is considered as a candidate for a more thorough search in Step 3, it is loaded.
Its AST is traversed and at each AST node we check if there is a cross reference to one of the Target URIs (Step 1).
If yes, the AST node is collected in the set of found references.
See class <code>ConcreteSyntaxAwareReferenceFinder</code> for implementation details.</p>
</div>
<div class="paragraph">
<p>The UI dependent logic may apply additional filters to drop references that are not relevant to the user, e.g. the reference from an AST element to its inferred type and vice versa (see <code>N4JSReferenceQueryExecutor.isToBeIgnored(EReference)</code>).</p>
</div>
</div>
<div class="sect4">
<h5 id="_limitations-and-possible-enhancements"><a class="anchor" href="#_limitations-and-possible-enhancements"></a><a class="link" href="#_limitations-and-possible-enhancements">11.6.2.5. Limitations and Possible Enhancements</a></h5>
<div class="paragraph">
<p>Other noteworthy limitations and potential enhancements of the current implementations are:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Semantics: Only references that are available in the model as real references are reported. Even though <code>getB()</code> in <code>myA.getB().getC()</code> may return an instance of type <code>B</code>, there is no reference reported to B in that expression, though a reference to a member of B would be reported for <code>getC</code>.</p>
</li>
<li>
<p>Visibility constraints are not applied and thus do not reduce the search scope to allow the report of invalidly established references in a later validation.</p>
</li>
</ul>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_compilation"><a class="anchor" href="#_compilation"></a><a class="link" href="#_compilation">12. Compilation</a></h2>
<div class="sectionbody">
<div class="admonitionblock warning">
<table>
<tr>
<td class="icon">
<i class="fa icon-warning" title=""></i>
</td>
<td class="content">
This chapter may be outdated.
</td>
</tr>
</table>
</div>
<div class="sect2 language-n4js">
<h3 id="chap:compilation"><a class="anchor" href="#chap:compilation"></a><a class="link" href="#chap:compilation">12.1. Introduction</a></h3>
<div class="paragraph">
<p>Compilation is the process of transforming some source code written by means of a human readable text into a machine readable target file, i.e. bytecode or assembler. However, in the context of N4JS, the target output is not machine code but another high-level programming language. This kind of compiler transforming from one programming language to another is called <em>transpiler</em>, as a combination of <code>transformation</code> and <code>compiler</code>. The design of the transpiler takes this special setup into account.</p>
</div>
<div class="sect3">
<h4 id="sec:general_design_rationals"><a class="anchor" href="#sec:general_design_rationals"></a><a class="link" href="#sec:general_design_rationals">12.1.1. General design rationals</a></h4>
<div class="sect4">
<h5 id="sec:logging_and_error_reporting"><a class="anchor" href="#sec:logging_and_error_reporting"></a><a class="link" href="#sec:logging_and_error_reporting">12.1.1.1. Logging and error reporting</a></h5>
<div class="paragraph">
<p>The generator always expects to get a valid model as an input. So all syntactic and semantic errors (not warnings) in the N4JS code as well as regarding the project configuration, e.g. in the package.json file, should be already discovered in the validation step. So any error marker on the resource will prevent the compiler to run.</p>
</div>
<div class="paragraph">
<p>In case of other errors arising in the generator, the error handling is done as follows:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>either throw an <code>GeneratorException</code> or better call <code>ExceptionHandler.handleError(message)</code> (that then will throw this exception)</p>
</li>
<li>
<p>beside the message also the file and the current line can be passed to <code>GeneratorException</code></p>
</li>
<li>
<p><code>GeneratorException</code> (as extending RuntimeException) will be handled by the generator caller</p>
<div class="ulist">
<ul>
<li>
<p>in UI: <code>BuildInstruction</code> will create an error log entry</p>
</li>
<li>
<p>headless: lets the <code>GeneratorException</code> fail</p>
</li>
</ul>
</div>
</li>
</ul>
</div>
</div>
<div class="sect4">
<h5 id="sec:progress_monitor"><a class="anchor" href="#sec:progress_monitor"></a><a class="link" href="#sec:progress_monitor">12.1.1.2. Progress monitor</a></h5>
<div class="paragraph">
<p>The compiler works on a single file and we do not expect that to take much time. Processors working on many files, such as linkers (in the JavaScript context, that is minification and concatenation), are different components.</p>
</div>
</div>
</div>
<div class="sect3">
<h4 id="sec:Xtext_Integration"><a class="anchor" href="#sec:Xtext_Integration"></a><a class="link" href="#sec:Xtext_Integration">12.1.2. Xtext Integration</a></h4>
<div class="sect4">
<h5 id="sec:xtext_default_behaviour"><a class="anchor" href="#sec:xtext_default_behaviour"></a><a class="link" href="#sec:xtext_default_behaviour">12.1.2.1. Xtext default behaviour</a></h5>
<div class="paragraph">
<p>The Xtext builder participant calculates the delta for every change in the project and triggers dirty state handling as well as code generation. By default the builder participant expects exactly one implementation bound to the IGenerator interface and thus only one output configuration provider belonging to this generator.</p>
</div>
</div>
<div class="sect4">
<h5 id="sec:n4js_requirements"><a class="anchor" href="#sec:n4js_requirements"></a><a class="link" href="#sec:n4js_requirements">12.1.2.2. N4JS requirements</a></h5>
<div class="paragraph">
<p>In constrast to the default Xtext behaviour in the N4JS IDE allows</p>
</div>
<div class="ulist">
<ul>
<li>
<p>the registration / discovery of multiple generators (including compilers and transpilers) but even no compiler at all, so that it is possible to ship the IDE also without any generators</p>
</li>
<li>
<p>to configure these generators separately (output paths and so on) workspace globally but also project specific (currently it is only required to enable / disable a compiler)</p>
</li>
<li>
<p>to enable / disable generators, but it is allowed to enable more than one compiler at one</p>
</li>
<li>
<p>to start compilers headless, i.e. e.g in automated build / JUnit tests but with the possibility to configure them there as well</p>
</li>
</ul>
</div>
</div>
<div class="sect4">
<h5 id="sec:compiler_discovery_in_ui"><a class="anchor" href="#sec:compiler_discovery_in_ui"></a><a class="link" href="#sec:compiler_discovery_in_ui">12.1.2.3. Compiler discovery in UI</a></h5>
<div class="ulist">
<ul>
<li>
<p>There is a singleton <code>ICompositeGenerator</code>. The <code>ICompositeGenerator</code> instance itself doesn’t contain generator logic but knows subgenerators (that implement <code>ISubGenerator</code>) to which it delegates the input file (so instead of registering a new builder participant (that then would have to recalculate the delta) only the generator call is extracted)</p>
</li>
<li>
<p>In the UI case, the actual execution of the registered generator is done in <code>BuildInstruction</code> where every N4JS resource contained in the delta calculated by the builder participant is an <code>ICompositeGenerator</code> implementation.</p>
</li>
<li>
<p><a href="#fig:cd_GeneratorAndBuilderParticipant">Builder (UI) and Generator (Core)</a> summarizes the implementation of this compiler infrastructure in the UI case.</p>
</li>
</ul>
</div>
<div id="fig:cd_GeneratorAndBuilderParticipant" class="imageblock center">
<div class="content">
<img src="chapters/13_compilation/images/cd_GeneratorAndBuilderParticipant.svg" alt="cd GeneratorAndBuilderParticipant">
</div>
<div class="title">Figure 25. Builder (UI) and Generator (Core)</div>
</div>
</div>
<div class="sect4">
<h5 id="sec:compiler_discovery_in_headless"><a class="anchor" href="#sec:compiler_discovery_in_headless"></a><a class="link" href="#sec:compiler_discovery_in_headless">12.1.2.4. Compiler discovery in headless</a></h5>
<div class="ulist">
<ul>
<li>
<p><code>N4JSHeadlessStandaloneSetup</code> is used to combine <code>N4JSRuntimeModule</code> and <code>N4JSHeadlessGeneratorModule</code> so there is <code>PropertiesFileBasedValuesProvider</code> is bind as implementation for <code>IPreferenceValuesProvider</code></p>
</li>
<li>
<p>via <code>N4JSHeadlessStandaloneSetup</code> the injector is created which is used to create an instance of <code>ICompositeGenerator</code></p>
</li>
<li>
<p>in the headless mode the subgenerators of are manually registered via an extension point in the class <code>HeadlessExtensionRegistrationHelper</code>.</p>
</li>
</ul>
</div>
</div>
<div class="sect4">
<h5 id="sec:general_generator_implementation"><a class="anchor" href="#sec:general_generator_implementation"></a><a class="link" href="#sec:general_generator_implementation">12.1.2.5. General generator implementation</a></h5>
<div class="exampleblock">
<div class="title">Example 1. Simple IGenerator</div>
<div class="content">
<div class="paragraph">
<p>The following snippet shows a minimal generator example.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-n4js" data-lang="n4js">public class Generator implements IGenerator {
@Override public void doGenerate(Resource resource, IFileSystemAccess fsa) {
final String filename = computeTargetFileName(resource.getURI());
Script script = IterableExtensions.&lt;Script&gt; head(
Iterables.&lt;Script&gt; filter(resource.getContents(), Script.class));
fsa.generateFile(filename, doCompileToString(script));
}
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>Generation is triggered for each Xtext resource. <code>IFileSystemAccess</code> is an abstraction of where to write the generated artefacts. This enables using the generator in the IDE (with a workspace) and headless (directly operating on files). In tests you can use <code>InMemoryFileAccess</code>, in standalone mode you should use <code>JavaFileSystemAccess</code> and in Eclipse mode <code>EclipseFileSystemAccess2</code></p>
</div>
</div>
</div>
</div>
<div class="sect4">
<h5 id="sec:general_generator_activation"><a class="anchor" href="#sec:general_generator_activation"></a><a class="link" href="#sec:general_generator_activation">12.1.2.6. General generator activation</a></h5>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Programmatically
</td>
<td class="hdlist2">
<p>Invoke the <code>IComposedGenerator.doGenerate</code> with a loaded N4JS resource and a configured <code>IFileSystemAccess</code> implementation.</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Xtext Builder
</td>
<td class="hdlist2">
<p>This is available by default when using the bound <code>IGenerator</code>, it runs on every change of your N4JS resource when automatic build is enabled in the workspace.</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Context Menu
</td>
<td class="hdlist2">
<p>see <a href="christiandietrich.wordpress.com/2011/10/15/xtext-calling-the-generator-from-a-context-menu/">Christian Dietrich’s Blog 2011/10/15</a></p>
</td>
</tr>
<tr>
<td class="hdlist1">
MWE2 Workflow
</td>
<td class="hdlist2">
<p>via <code>org.eclipse.xtext.generator.GeneratorComponent</code> that is configured with the standalone setup of the N4JS language. Such an MWE2 workflow also requires the <code>org.eclipse.xtext.mwe.Reader</code> component to first load the N4JS resources to transform in a model slot that is then consumed by the GeneratorComponent</p>
</td>
</tr>
</table>
</div>
</div>
</div>
<div class="sect3">
<h4 id="sec:Overview_of_Input_Models"><a class="anchor" href="#sec:Overview_of_Input_Models"></a><a class="link" href="#sec:Overview_of_Input_Models">12.1.3. Overview of the Input Models</a></h4>
<div class="paragraph">
<p>The input is a simple instance of <code>Script</code>, which is the root model element for all N4JS files. Actually, it is the root of the AST. For the AST elements, other elements stemming from other models are accessible as well. The following models may be important for the compiler:</p>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
N4JS AST
</td>
<td class="hdlist2">
<p>The abstract syntax tree is an EMF model, it is defined in a single Xcore file <code>N4JS.xcore</code></p>
</td>
</tr>
<tr>
<td class="hdlist1">
Parser or Node Model
</td>
<td class="hdlist2">
<p>The parser tree, also called node model, is defined by Xtext. It contains offset information, holding whitespaces, line breaks, comments as well as other hidden tokens. It can be accessed via <code>NodelModelUtils</code>, that is a node can be retrieved for a given AST element and vice versa. There are three different kind of nodes: root, composite and leaf node.<br>
<strong>As of Dec 2015, the transpiler does no longer make use of the parse tree!</strong></p>
</td>
</tr>
<tr>
<td class="hdlist1">
Type Model
</td>
<td class="hdlist2">
<p>The type model is an abstract view on the N4JS AST. It is defined in a single Xcore file <code>Types.xcore</code>. Not all AST elements are related to type model information. This is only true for subtypes of <code>TypeDefiningElement</code>, with references to <code>Type</code> or containing a <code>TypeRef</code>.</p>
</td>
</tr>
<tr>
<td class="hdlist1">
N4 Project
</td>
<td class="hdlist2">
<p>via <code>OutputPathHelper</code> located in <code>org.eclipse.n4js.generator</code> wraps the calculation of compiled file path.</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Grammar Model
</td>
<td class="hdlist2">
<p>Grammar Model created from <code>N4JS.xtext</code>, the rules can be access in the Java code via <code>N4JSGrammarAccess</code>. The grammar elements can be retrieved from the parser model vial <code>node.getGrammarElement()</code>. <code>org.eclipse.xtext.GrammarUtil</code> also contains some useful helper methods.<br>
<strong>As of Dec 2015, the transpiler does no longer make use of the grammar model!</strong></p>
</td>
</tr>
</table>
</div>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Core_Generator"><a class="anchor" href="#sec:Core_Generator"></a><a class="link" href="#sec:Core_Generator">12.2. Generators</a></h3>
<div class="paragraph">
<p>Generators are an abstraction above that of transpilers. N4JS transpilers are implemented as specific generators, but there might be other generators that are not transpilers (e.g. generator that produces HTML documentation from the jsdoc in the N4JS source files).</p>
</div>
<div class="sect3">
<h4 id="sec:Compiler_Components"><a class="anchor" href="#sec:Compiler_Components"></a><a class="link" href="#sec:Compiler_Components">12.2.1. Generator Components</a></h4>
<div class="paragraph">
<p><a href="#fig:comp_compilers">Compiler Components</a> gives an overview over the compiler related components. Some of these components are described in detail in the following sections.
As of Dec 2017, the generator architecture has been refactored and simplified.</p>
</div>
<div class="ulist">
<ul>
<li>
<p>There is only a single <code>ICompositeGenerator</code> instance. Since the single instance should simply delegate to subgenerators, composite generators can no longer be registered via extension point.</p>
</li>
<li>
<p>Most of generator related code is moved into <code>org.eclipse.n4js</code> bundle. This is needed because we need to bind <code>ICompositeGenerator</code> to a concrete implementation in the <code>org.eclipse.n4js</code> bundle and the extension point for <code>ICompositeGenerator</code> has been removed.</p>
</li>
<li>
<p>An extension point <code>org.eclipse.n4js.generator.subgenerator</code> is introduced in the <code>org.eclipse.n4js</code> bundle. This makes it possible to register a new subgenerator via extension point.</p>
</li>
</ul>
</div>
<div id="fig:comp_compilers" class="imageblock center">
<div class="content">
<img src="chapters/13_compilation/images/comp_compilers.svg" alt="comp compilers">
</div>
<div class="title">Figure 26. Compiler Components</div>
</div>
<div class="paragraph">
<p><a href="#fig:od_generatorInjection">Discovering generators and provide them with Guice bindings.</a> shows how composite generator and subgenerators interact with other components both in the UI and in the headless case.</p>
</div>
<div id="fig:od_generatorInjection" class="imageblock center">
<div class="content">
<img src="chapters/13_compilation/images/od_generatorInjection.svg" alt="od generatorInjection">
</div>
<div class="title">Figure 27. Discovering generators and provide them with Guice bindings.</div>
</div>
<div class="paragraph">
<p>As we can see in the diagram above. In the UI case, <code>N4JSBuilderParticipant</code> creates <code>BuildInstruction</code> which in turn delegates the generation logics to an instance of <code>ICompositeGenerator</code>. The <code>ICompositeGenerator</code> simply delegates the generation logics to subgenerators .</p>
</div>
<div class="paragraph">
<p>In the headless mode, <code>n4jscBase.doMain</code> creates an instance of <code>N4JSStandaloneSetup</code> and obtains the injector from there. This injector is then used to create an instance of <code>ICompositeGenerator</code> in <code>N4HeadlessCompiler</code>.</p>
</div>
</div>
<div class="sect3">
<h4 id="sec:Generator_architecture"><a class="anchor" href="#sec:Generator_architecture"></a><a class="link" href="#sec:Generator_architecture">12.2.2. Generator architecture</a></h4>
<div class="paragraph">
<p>The compiler has to create different compilation targets, e.g., for web applications running in a browser (Chrome), or for applications running on iOS using the JavaScriptCore framework <sup class="footnote">[<a id="_footnoteref_13" class="footnote" href="#_footnote_13" title="View footnote.">13</a>]</sup>. Other scenarios may include code created for debugging purposes vs. optimized code, although this may be implementable via configuration switches as well.</p>
</div>
<div class="paragraph">
<p><a href="#fig:cd_SubGenerators">Generator and sub-generators</a> shows the main generator classes, including two sub generators for EcmaScript code and EcmaScript on iOS.</p>
</div>
<div id="fig:cd_SubGenerators" class="imageblock center">
<div class="content">
<img src="chapters/13_compilation/images/cd_SubGenerators.svg" alt="cd SubGenerators">
</div>
<div class="title">Figure 28. Generator and sub-generators</div>
</div>
</div>
<div class="sect3">
<h4 id="sec:Unified_Compiler_Configuration"><a class="anchor" href="#sec:Unified_Compiler_Configuration"></a><a class="link" href="#sec:Unified_Compiler_Configuration">12.2.3. Unified Compiler Configuration</a></h4>
<div class="paragraph">
<p>Since the compiler is to be used in both UI and headless (or CLI) mode, the configuration has to abstract from Eclipse <code>IPreferenceStore</code> concept or CLI utility classes. This is done with the combination of <code>CompilerDescriptor</code> and <code>CompilerProperties</code>, used by all <code>ISubGenerator</code> implementations (see <a href="#fig:cd_SubGenerators">Fig. Sub Generators</a>).</p>
</div>
<div class="paragraph">
<p>Each compiler provides</p>
</div>
<div class="ulist">
<ul>
<li>
<p>a unique name (that have to match with the name of the output configuration)</p>
</li>
<li>
<p>a default compiler descriptor that contains the preference values to be applied when nothing else been configured in the provided preference values</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>A <code>CompilerDescriptor</code> has</p>
</div>
<div class="ulist">
<ul>
<li>
<p>an identifier (this is the unique name of the compiler as mentioned before)</p>
</li>
<li>
<p>a name (a readable name to used in Eclipse preference page)</p>
</li>
<li>
<p>a description (not used yet, but maybe later also shown in the preference page)</p>
</li>
<li>
<p>a flag, that indicates, if this generator should run by default</p>
</li>
<li>
<p>the file extension to be used for the compiled file</p>
</li>
<li>
<p>the <code>OutputConfiguration</code> object from Xtext that contains output related preferences like the output folder</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>The <code>CompilerProperties</code> is an enumeration that makes it easier to iterate over the preferences and getting / setting the preference values in a generic way. So this enumeration contains all configurable properties as literals.</p>
</div>
<div class="paragraph">
<p>The keys for preferences have to follow a fixed pattern as it also used internally by the builder participant when applying the configurations from the <code>OutputConfiguration</code>. So the key consists of</p>
</div>
<div class="ulist">
<ul>
<li>
<p>’outlet’</p>
</li>
<li>
<p>unique name of the compiler = unique name of the output configuration</p>
</li>
<li>
<p>the name of the property</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Example: outlet.es5.compiledFileExtension</p>
</div>
<div class="paragraph">
<p><code>N4JSPreferenceAccess</code> encapsulates the access to the injected <code>IPreferenceValuesProvider</code>. This values provider is bound in UI to <code>EclipsePreferencesProvider</code> that creates an overlay over the default configuration and makes it so possible to have workspace global as well as project specific preferences and always as fall back the default values.</p>
</div>
<div class="paragraph">
<p>In headless mode the <code>PropertiesFileBasedValuesProvider</code> is bound as implementation of <code>IPreferenceValuesProvider</code>. With this implementation it is possible to load the preferences from a provided properties file.</p>
</div>
<div class="paragraph">
<p><code>N4JSPreferenceAccess</code> is used in <code>AbstractSubGenerator</code> which provided the most common used preferences as extra methods.</p>
</div>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Transpilers"><a class="anchor" href="#sec:Transpilers"></a><a class="link" href="#sec:Transpilers">12.3. Transpilers</a></h3>
<div class="paragraph">
<p>Transpilers are a special case of generators, used for transforming N4JS source code into some target code in some other, high-level programming language. In this section we describe the general transpiler infrastructure without considering any particular transpiler. Currently, there is only a single such concrete transpiler for ECMAScript target code, explained later in <a href="#sec:N4JS_to_EcmaScript_Transpiler">N4JS-to-EcmaScript Transpiler</a>.</p>
</div>
<div class="paragraph">
<p>All code of the general transpiler infrastructure is found in bundle <code>org.eclipse.n4js.transpiler</code>.</p>
</div>
<div class="sect3">
<h4 id="sec:Phases"><a class="anchor" href="#sec:Phases"></a><a class="link" href="#sec:Phases">12.3.1. Overview</a></h4>
<div class="paragraph">
<p><a href="#fig:ad_PipelineOverview">Overview of the compilation pipeline</a> shows an overview of the steps during transpilation of a single resource:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>an initial conversion from the original AST to an <strong>intermediate model (IM)</strong>, called <strong>preparation step</strong>.</p>
</li>
<li>
<p>one or more <strong>transformation</strong> phases, each taking as input the IM and performing a number of in-place modification on it.</p>
</li>
<li>
<p>a final <strong>pretty printing step</strong> that transform the final version of the IM into the textual output, i.e. the target code.</p>
</li>
</ol>
</div>
<div id="fig:ad_PipelineOverview" class="imageblock center">
<div class="content">
<img src="chapters/13_compilation/images/ad_PipelineOverview.svg" alt="ad PipelineOverview">
</div>
<div class="title">Figure 29. Overview of the compilation pipeline</div>
</div>
<div class="paragraph">
<p>The IM is the most important data structure in the transpiler. It starts out as a 1-to-1 copy of the original AST and is then gradually transformed by the AST transformation steps into, ultimately, a representation of the output code. Only the IM undergoes updates, while the original AST remains unchanged. Nodes in the IM that are identical on N4JS source code and target code side can simply be left unchanged. Traceability links allow navigating back to an original AST node from a given IM node, but due to the gradual modification of the IM this might not be possible for all IM nodes (the tracer will return <code>null</code> in those cases.</p>
</div>
<div class="paragraph">
<p>Ideally, each transformation executed during the transformation step should be self-contained and coupling should be reduced to a minimum. Of course, this is not possible in all cases, in practice. Therefore, a simple mechanism is provided for statically specifying dependencies between transformations by way of Java annotations (see Java class <code>TransformationDependency</code> for more details). The ECMAScript transpiler, for example, has 18 individual transformations (at time of writing).</p>
</div>
</div>
<div class="sect3">
<h4 id="relation-between-ast-and-im"><a class="anchor" href="#relation-between-ast-and-im"></a><a class="link" href="#relation-between-ast-and-im">12.3.2. Relation between AST and IM</a></h4>
<div class="paragraph">
<p>The relation between the original AST and the IM is maintained by the tracer, see class <code>Tracer</code>, which is available via the transpiler state. The tracer allows to obtain a IM elements for a given original AST node and, conversely, original AST nodes for a given IM element (i.e. a 1:N association between original AST node and IM element).</p>
</div>
<div class="paragraph">
<p>The main purpose of this tracing information is to compute source maps for the target code.</p>
</div>
<div class="paragraph">
<p>At the beginning of the transformation step, there is a 1-to-1 correspondence between AST and IM, but over the course of the transformations this correspondence will become more and more blurred. Therefore, whenever using the tracer to get to the original AST from a given IM element , we have to consider the case that there is not original AST node defined for (because was created programmatically by an earlier transformation) OR that the original AST node is of a different kind than (because, maybe, an original N4JS class declaration was replaced by a function declaration by an earlier transformation).</p>
</div>
<div class="paragraph">
<p>Whenever a transformation changes the IM, it is responsible to update the tracer, accordingly.</p>
</div>
</div>
<div class="sect3">
<h4 id="implementation-overview"><a class="anchor" href="#implementation-overview"></a><a class="link" href="#implementation-overview">12.3.3. Implementation Overview</a></h4>
<div class="paragraph">
<p><a href="#fig:transpilerClassDgr">Transpiler Class Diagram</a> shows a class diagram of the main constituents of the transpiler infrastructure.</p>
</div>
<div id="fig:transpilerClassDgr" class="imageblock center">
<div class="content">
<img src="chapters/13_compilation/images/TranspilerClassDgr.svg" alt="TranspilerClassDgr">
</div>
<div class="title">Figure 30. Class diagram for the transpiler infrastructure.</div>
</div>
<div class="paragraph">
<p>The <code>AbstractTranspiler</code> controls the overall workflow shown earlier in . Concrete subclasses of <code>Transformation</code> perform the actual transformations (the preparation and pretty-printing steps are not shown in the above class diagram). Concrete transformations are created via injection within concrete sub classes of <code>AbstractTranspiler</code> (see class <code>EcmaScriptTranspiler</code> for an example). All information required during transpilation is kept in a simple data class called <code>TranspilerState</code>; a single instance of this class is created during the preparation step and is passed along until transpilation of the resource to transpile is completed.</p>
</div>
<div class="paragraph">
<p>Class <code>Transformation</code> has a super class <code>TranspilerComponent</code> that has two important responsibilities:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>it contains many utility methods that are easily accessible from within concrete transformations through inheritance.</p>
</li>
<li>
<p>it obtains the transpiler state via injection (using the scoping feature of Google Guice, for more details see <code>org.eclipse.n4js.utils.di.scopes.ScopeManager</code> and <code>TransformationScoped</code>). This injection is done in super class <code>TranspilerComponent</code>, so when implementing a new transformation, the programmer does not have to deal with these details and can simply obtain the transpiler state via the inherited method <code>TranspilerComponent#getState()</code>.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Code shared across concrete transformations should be placed in sub classes of <code>TransformationAssistant</code>. Those assistants are similar to the helpers used elsewhere, but by sharing the <code>TranspilerComponent</code> super class they get all the utility methods provided by that class and they automatically get the transpiler state.</p>
</div>
<div class="paragraph">
<p>For more implementation details see the code and javadoc; a good starting point for investigating the overall workflow are classes <code>AbstractTranspiler</code> and <code>Transformation</code>.</p>
</div>
</div>
<div class="sect3">
<h4 id="sec:Guidelines_for_Implementing_Transformations"><a class="anchor" href="#sec:Guidelines_for_Implementing_Transformations"></a><a class="link" href="#sec:Guidelines_for_Implementing_Transformations">12.3.4. Guidelines for Implementing Transformations</a></h4>
<div class="paragraph">
<p>Some hints:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>if you need to create an entirely new transformation:</p>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>create new sub class of <code>Transformation</code> (use Xtend).</p>
</li>
<li>
<p>in the main class of the transpiler you are working with (probably <code>EcmaScriptTranspiler</code>), change method
<code>#computeTransformationsToBeExecuted()</code> to return an instance of your new transformation. The instance should be created using a Guice provider (see <code>EcmaScriptTranspiler</code> for an example). Note that this method also defines the order of transformations!</p>
</li>
<li>
<p>implement the <code>#transform()</code> method of your newly created transformation.</p>
</li>
<li>
<p>consider adding pre and post conditions via methods <code>#assertPreConditions()</code> and <code>#assertPostConditions()</code> (throw an AssertionError if failed).</p>
</li>
<li>
<p>consider declaring dependencies to other transformations using the annotations defined in class <code>TransformationDependency</code>.</p>
</li>
</ol>
</div>
</li>
<li>
<p>code shared across transformations should be placed in a new or existing sub class of <code>TransformationAssistant</code> and then this assistant should be injected into the transformations that require this code’s functionality.</p>
</li>
<li>
<p>inside a transformation or transformation assistant:</p>
<div class="ulist">
<ul>
<li>
<p>to modify the IM, use the utility methods inherited from <code>TranspilerComponent</code> (e.g. <code>#replace()</code>, <code>#insertBefore()</code>); try to avoid direct manipulation of the IM as far as possible (but sometimes it’s necessary).</p>
</li>
<li>
<p>to create new IM elements, use the convenience methods in <code>TranspilerBuilderBlocks</code>; use static import.</p>
</li>
<li>
<p>to create a new symbol table entry or to obtain an existing symbol table entry for a given original target or element in the IM, use the inherited utility methods <code>TranspilerComponent#getSymbolTableEntry*()</code>.<br>
<strong>Never search or modify the symbol table directly!</strong></p>
</li>
<li>
<p>to access the transpiler state <sup class="footnote">[<a id="_footnoteref_14" class="footnote" href="#_footnote_14" title="View footnote.">14</a>]</sup>, use inherited method <code>TranspilerComponent#getState()</code> (by convention, in Xtend you should just write <code>state</code> as if it were a field).</p>
</li>
</ul>
</div>
</li>
<li>
<p>for local testing, activate additional consistency checks between transformations and assertion of pre/post conditions via these boolean flags:<br>
<code>AbstractTranspiler#DEBUG_PERFORM_VALIDATIONS</code>,<br>
<code>AbstractTranspiler#DEBUG_PERFORM_ASSERTIONS</code>.</p>
</li>
<li>
<p>never add one of the following replaced EMF entities to the IM:<br>
<code>Script</code>,<br>
<code>IdentifierRef</code>,<br>
<code>ParameterizedTypeRef</code>,<br>
<code>ParameterizedTypeRefStructural</code>,<br>
<code>ParameterizedPropertyAccessExpression</code>.<br>
Instead, use the replacement entities from <code>IM.xcore</code> that have the <code>_IM</code> suffix (e.g. <code>IdentifierRef_IM</code>). If you always use <code>TranspilerBuilderBlocks</code> as described above, you won’t run into this issue.</p>
</li>
</ul>
</div>
</div>
<div class="sect3">
<h4 id="symbol-table-in-the-im"><a class="anchor" href="#symbol-table-in-the-im"></a><a class="link" href="#symbol-table-in-the-im">12.3.5. Symbol Table in the IM</a></h4>
<div class="paragraph">
<p>During the preparation step, the IM is created as an exact copy of the original AST in most cases. However, to make sure the IM is self-contained and does not have any cross-references to the original AST or the original TModule and to simplify certain computations within the transformations, some AST entities are modified. For this purpose, there is a small EMF model called <code>IM.xcore</code>. It extends the AST model <code>n4js.xcore</code> and adds some elements.</p>
</div>
<div class="paragraph">
<p>Most importantly, a symbol table is created and all references of the original AST pointing to an IdentifiableElement (either in the original AST or in the TModule) are rewired to a reference to an entry in the symbol table. Those entries are of type <code>SymbolTableEntry</code> and occur in three special forms (there is a dedicated sub class for each case). Detailed information is provided in the javadoc of <code>SymbolTableEntry</code> and its sub classes and is not repeated here to avoid duplication.</p>
</div>
<div class="paragraph">
<p>The following entity replacements are done while creating the IM from the original AST and the entities without <code>_IM</code> must <strong>never</strong> appear in the IM:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>Script</code> <code>Script_IM</code></p>
</li>
<li>
<p><code>IdentifierRef</code> <code>IdentifierRef_IM</code></p>
</li>
<li>
<p><code>ParameterizedTypeRef</code> <code>ParameterizedTypeRef_IM</code></p>
</li>
<li>
<p><code>ParameterizedTypeRefStructural</code> <code>ParameterizedTypeRefStructural_IM</code></p>
</li>
<li>
<p><code>ParameterizedPropertyAccessExpression</code> <code>ParameterizedPropertyAccessExpression_IM</code></p>
</li>
</ul>
</div>
<div class="paragraph">
<p>For example, when having in the original AST an <code>IdentifierRef</code> pointing to identifiable element , then the IM will contain an <code>IdentifierRef_IM</code> pointing to a <code>SymbolTableEntryOriginal</code> with a property <code>originalTarget</code> pointing to .</p>
</div>
<div class="paragraph">
<p>Figures <a href="#fig:rewire_var">Rewire Var</a>, <a href="#fig:rewire_class">Rewire Class</a>, and <a href="#fig:rewire_import">Rewire Import</a> show a comparison between an original AST with its original TModule and the self-contained intermediate model for a number of concrete examples.</p>
</div>
<div class="exampleblock">
<div class="title">Example 2. Intermediate Models for References to Variables</div>
<div class="content">
<div class="paragraph">
<p>Original AST + TModule</p>
</div>
<div id="fig:rewire_var" class="imageblock center">
<div class="content">
<img src="chapters/13_compilation/images/Rewire_var_pre.png" alt="Rewire var pre">
</div>
<div class="title">Figure 31. Intermediate Model for References to Variables</div>
</div>
<div class="paragraph">
<p>Intermediate model (IM)</p>
</div>
<div id="fig:rewire_var-post" class="imageblock center">
<div class="content">
<img src="chapters/13_compilation/images/Rewire_var_post.png" alt="Rewire var post">
</div>
<div class="title">Figure 32. Intermediate Model for References to Variables (post)</div>
</div>
</div>
</div>
<div class="exampleblock">
<div class="title">Example 3. Intermediate Model for References to Classes</div>
<div class="content">
<div class="paragraph">
<p>original AST + TModule</p>
</div>
<div id="fig:rewire_class" class="imageblock center">
<div class="content">
<img src="chapters/13_compilation/images/Rewire_class_pre.png" alt="Rewire class pre">
</div>
<div class="title">Figure 33. Intermediate Model for References to Classes</div>
</div>
<div class="paragraph">
<p>Intermediate model (IM)</p>
</div>
<div id="fig:rewire_class-post" class="imageblock center">
<div class="content">
<img src="chapters/13_compilation/images/Rewire_class_post.png" alt="Rewire class post">
</div>
<div class="title">Figure 34. Intermediate Model for References to Classes (post)</div>
</div>
</div>
</div>
<div class="exampleblock">
<div class="title">Example 4. Intermediate Model for References to Imported Classes</div>
<div class="content">
<div class="paragraph">
<p>Original AST + TModule</p>
</div>
<div id="fig:rewire_import" class="imageblock center">
<div class="content">
<img src="chapters/13_compilation/images/Rewire_import_pre.png" alt="Rewire import pre">
</div>
<div class="title">Figure 35. Intermediate Model for References to Imported Classes</div>
</div>
<div class="paragraph">
<p>Intermediate model (IM)</p>
</div>
<div id="fig:rewire_import-post" class="imageblock center">
<div class="content">
<img src="chapters/13_compilation/images/Rewire_import_post.png" alt="Rewire import post">
</div>
<div class="title">Figure 36. Intermediate Model for References to Imported Classes (post)</div>
</div>
</div>
</div>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:N4JS_to_EcmaScript_Transpiler"><a class="anchor" href="#sec:N4JS_to_EcmaScript_Transpiler"></a><a class="link" href="#sec:N4JS_to_EcmaScript_Transpiler">12.4. N4JS-to-EcmaScript Transpiler</a></h3>
<div class="sect3">
<h4 id="sec:Overview_of_Transformations"><a class="anchor" href="#sec:Overview_of_Transformations"></a><a class="link" href="#sec:Overview_of_Transformations">12.4.1. Overview of Transformations</a></h4>
<div class="paragraph">
<p>The following overview will soon be outdated. Therefore:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>to find out which transformations are actually being executed and in what precise order, it is best to directly look into method:<br>
<code>EcmaScriptTranspiler#computeTransformationsToBeExecuted()</code>.</p>
</li>
<li>
<p>to learn about dependencies between transformations, check the annotations of the transformation class to see if one of the dependency annotations defined in <code>TransformationDependency</code> are given there (though probably not all dependencies will be specified in that form).</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>The following table lists all transformation by class name in the order they are executed by the <code>EcmaScriptTranspiler</code>.</p>
</div>
<table class="tableblock frame-all grid-all spread">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">StaticPolyfillTransformation</p></td>
<td class="tableblock halign-left valign-top"></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">MemberPatchingTransformation</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">see <a href="#sec:Transpiling_members">Transpiling Members</a></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">ApiImplStubGenerationTransformation</p></td>
<td class="tableblock halign-left valign-top"></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">DestructuringTransformation</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">turn destructuring patterns into ES5 code</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">SuperLiteralTransformation</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">super call + super access</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">ExpressionTransformation</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">casts, <code>instanceof</code>, <code>@Promisify</code>, &#8230;&#8203;</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">DependencyInjectionTransformation</p></td>
<td class="tableblock halign-left valign-top"></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">ClassDeclarationTransformation</p></td>
<td class="tableblock halign-left valign-top"></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">InterfaceDeclarationTransformation</p></td>
<td class="tableblock halign-left valign-top"></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">EnumDeclarationTransformation</p></td>
<td class="tableblock halign-left valign-top"></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">FunctionDeclarationTransformation</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">turn declared function into variable declaration + function expression</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">ArrowFunction_Part1_Transformation</p></td>
<td class="tableblock halign-left valign-top"></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">BlockTransformation</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">local arguments variable, <code>await</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">FormalParameterTransformation</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">variadic arguments</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">ArrowFunction_Part2_Transformation</p></td>
<td class="tableblock halign-left valign-top"></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">TrimTransformation</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">remove TypeRefs and TypeVariables</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">SanitizeImportsTransformation</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">remove unused imports + add missing imports</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">ModuleWrappingTransformation</p></td>
<td class="tableblock halign-left valign-top"></td>
</tr>
</tbody>
</table>
<div class="paragraph">
<p>The main complexity lies in the three transformations for N4JS type declarations (classes, interfaces, enums) and the related three transformations for member handling at the beginning (static polyfills, member patching, API/Impl stub generation) and the module wrapping. Up to the double horizontal line, the IM is still rather close to N4JS (e.g. still contains <code>N4ClassDeclaration</code>s with <code>N4MemberDeclaration</code>s), but after that it rapidly departs from the structure of the original AST (e.g. class declarations are broken up into a function declaration and a $<code>makeClass</code> call, field accessors and methods become function expressions in the properties of an object literal, fields are handled differently).</p>
</div>
</div>
<div class="sect3">
<h4 id="sec:Transpiling_members"><a class="anchor" href="#sec:Transpiling_members"></a><a class="link" href="#sec:Transpiling_members">12.4.2. Transpiling members</a></h4>
<div class="paragraph">
<p>When processing the members of a container type, in the standard case, the transpiler simply has to generate target code for each owned member. For inherited members no output code has to be generated, because the ordinary semantics of the Javascript prototype chain is used in the generated code.</p>
</div>
<div class="paragraph">
<p>There are, however, special cases when output code has to be generated for a non-owned or non-existant member of a container type:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>partial shadowing caused by lone field accessors, [sec:Transpiling_members__Partial_shadowing_of_getter_setter_pairs]<br>
( <strong>delegation</strong>)</p>
</li>
<li>
<p>consumption of members of an interface within an implementing class, [sec:Transpiling_members__Consuming_or_inheriting_members_of_an_interface]<br>
( <strong>delegation</strong>, for data fields: <strong>copying</strong>)</p>
</li>
<li>
<p>inheritance of members of an interface within an extending interface, [sec:Transpiling_members__Consuming_or_inheriting_members_of_an_interface]<br>
( <strong>delegation</strong>, for data fields: <strong>copying</strong>)</p>
</li>
<li>
<p>mixing in members into a container type via static polyfill, [sec:Transpiling_members__Static_polyfill]<br>
( <strong>copying</strong>)</p>
</li>
<li>
<p>adding an API / implementation stub, [sec:Transpiling_members__API_implementation_stubs]<br>
( <strong>creation</strong>)</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>The above overview also states what technique is used in each special case of member handling: <strong>delegation</strong>, <strong>copying</strong> or <strong>creation</strong>. Delegation is the most tricky one and means that not a new function is generated in the output code for the special member, but the existing member function of an existing member is obtained from somewhere in the prototype chain and used directly as the member function of the special member. <strong>Copying</strong> means that an existing member is copied to another location where the special handling is required as if it were defined in that place. Lastly, <strong>creation</strong> means that an entirely new member is created for which no existing member serves as a template and this member gets a body with some <code>default</code> behavior. These three techniques of special member handling are explained in more detail in <a href="#sec:Transpiling_members__Delegating_members">Techniques for special member handling</a>.</p>
</div>
<div class="sect4">
<h5 id="sec:Transpiling_members__Delegating_members"><a class="anchor" href="#sec:Transpiling_members__Delegating_members"></a><a class="link" href="#sec:Transpiling_members__Delegating_members">12.4.2.1. Techniques for special member handling</a></h5>
<div class="paragraph">
<p>If output code has to be generated for a non-owned member of a classifier we distinguish the following two cases:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>either some other member owned by classifier serves as a template for ,</p>
</li>
<li>
<p>or no such template exists.</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>In the first case, we can either <strong>copy</strong> in the sense that we will generate output code for within the output code for as if had been defined in . Or we can use <strong>delegation</strong>, i.e. generate output code for that reuses the existing member function of in . In case no template exists, we always have to <strong>create</strong> from scratch, i.e. generate output code as if had been defined with some behavior pre-defined by the N4JS language (this applies only to API / implementation stubs where this pre-defined behaviour is to throw an <code>unimplemented member</code> error).</p>
</div>
<div class="paragraph">
<p>Creation and copying is straightforward; for more details on member delegation see class <code>DelegationAssistant</code> and entity <code>DelegatingMember</code> in <code>IM.xcore</code>. The basic approach is to allow one transformation to create a <code>DelegatingMember</code> and insert it into the IM and let the transformations for class and interface declarations turn this member into low-level Javascript constructs that perform the actual delegation.</p>
</div>
</div>
<div class="sect4">
<h5 id="sec:Transpiling_members__Partial_shadowing_of_getter_setter_pairs"><a class="anchor" href="#sec:Transpiling_members__Partial_shadowing_of_getter_setter_pairs"></a><a class="link" href="#sec:Transpiling_members__Partial_shadowing_of_getter_setter_pairs">12.4.2.2. Partial shadowing</a></h5>
<div class="paragraph">
<p>In Javascript, if an object has a setter of name , then a read access <code>obj.prop</code> will return undefined, even if the prototype of has a getter or field of name . Conversely, if has a getter , then a write access <code>obj.prop = 42</code> will produce a runtime error, even if the prototype of has a setter or field .</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>var proto = {
get prop1() { return "this won't show up" },
set prop2(value) { console.log("this won't be reached") }
}
var obj = {
set prop1(value) {},
get prop2() {}
}
obj.__proto__ = proto;
console.log(typeof obj.prop1); // will print "undefined"
obj.prop2 = 42; // error: "setting a property that has only a getter"</code></pre>
</div>
</div>
<div class="paragraph">
<p>Note, in plain N4JS a validation enforces a redefinition of accessors or overriding of a field always by getter/setter pairs. However, in special situations of incomplete API implementations stubs for missing accessors are created in order to provide meaningful test-reporting. This leads to situations where on the implementation side a single getter or or a single setter is defined in a subclass - unaware of possibly injected stubs in superclasses. The aforementioned validation can not enforce the user to author an accessor pair. To keep a meaningful test-response the transpiler treats this situation as follows:</p>
</div>
</div>
<div class="sect4">
<h5 id="sec:Transpiling_members__Consuming_or_inheriting_members_of_an_interface"><a class="anchor" href="#sec:Transpiling_members__Consuming_or_inheriting_members_of_an_interface"></a><a class="link" href="#sec:Transpiling_members__Consuming_or_inheriting_members_of_an_interface">12.4.2.3. Consuming or inheriting members of an interface</a></h5>
<div class="paragraph">
<p>When an N4JS class consumes the member of an interface implemented by , then this cannot be handled by the native prototype chain mechanism of Javascript. Instead, the transpiler has to generate a member of corresponding type that delegates to the consumed member. In case of data fields, such a delegation is not possible and thus the transpiler generates output code for the consumed data field as if the field had been defined in .</p>
</div>
<div class="paragraph">
<p>Of particular importance in this context is the diamond problem when consuming members from an interface. For example, if interface defined method with a default implementation, interface extends and overrides with a different implementation, class implements and class extending implements , then will not consume because it has already inherited from its super class (which in turn has consumed it from ). So, in the default implementation of given in will be active, not that given in .</p>
</div>
</div>
<div class="sect4">
<h5 id="sec:Transpiling_members__Static_polyfill"><a class="anchor" href="#sec:Transpiling_members__Static_polyfill"></a><a class="link" href="#sec:Transpiling_members__Static_polyfill">12.4.2.4. Static polyfill</a></h5>
<div class="paragraph">
<p>See class <code>StaticPolyfillTransformation</code> for details.</p>
</div>
</div>
<div class="sect4">
<h5 id="sec:Transpiling_members__API_implementation_stubs"><a class="anchor" href="#sec:Transpiling_members__API_implementation_stubs"></a><a class="link" href="#sec:Transpiling_members__API_implementation_stubs">12.4.2.5. API / implementation stubs</a></h5>
<div class="paragraph">
<p>See <a href="#sec:Support_for_incomplete_API_implementation_testing_in_the_N4JS_to_EcmaScript_5_Transpiler">Support for incomplete API implementation testing</a>.</p>
</div>
</div>
</div>
<div class="sect3">
<h4 id="sec:Support_for_incomplete_API_implementation_testing_in_the_N4JS_to_EcmaScript_5_Transpiler"><a class="anchor" href="#sec:Support_for_incomplete_API_implementation_testing_in_the_N4JS_to_EcmaScript_5_Transpiler"></a><a class="link" href="#sec:Support_for_incomplete_API_implementation_testing_in_the_N4JS_to_EcmaScript_5_Transpiler">12.4.3. Support for incomplete API implementation testing</a></h4>
<div class="paragraph">
<p>As part of the introduction of API projects with executable test cases the need to verify the state of implementations came into focus. No formal dependency is allowed between an API project and its dedicated implementation projects, hence an inconsistency can not directly be detected. However at runtime (c.f. <a href="#_execution">Execution</a>) the API is always replaced by an appropriate implementation.</p>
</div>
<div class="paragraph">
<p>In cases where such an implementation is incomplete this would result in failures due to missing concepts, e.g. calls to methods that are not in place or usage of fields which are not defined. In order to support the author of an implementation the IDE provides a mean to compare the current state of implementation to the developer in a tabular way (c.f. [<a href="#N4JSSpec">N4JSSpec</a>]).</p>
</div>
<div class="paragraph">
<p>The key idea for automated test-support is to incorporate those comparison-results into the transpiled output in way, that a test-framework can easily distinguish wrong implementations from incomplete implementations. Of course this is not always possible, but the majority of cases can be handled.</p>
</div>
<div class="paragraph">
<p>As there is only one transpilation mode the resulting modifications are always part of the generated code.</p>
</div>
<div class="paragraph">
<p>In order to distinguish the different project-types we distinguish between different project types:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>an API-project (API)</p>
</li>
<li>
<p>an API-implementation project (Impl)</p>
</li>
<li>
<p>a client project (Client)</p>
</li>
<li>
<p>a test project (Test)</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>The API-project defines the requirements to it’s implementors in form of definition-files (n4jsd). The API is defined together with an test-project which validates the implementation. Client code is written with a formal dependency to the API and uses the elements declared therein. In that sense an API-testing project is just a normal client project with a test-nature.</p>
</div>
<div class="paragraph">
<p>Additional code to support API implementation testing is only inserted into the Impl-projects. API, Client and Test are not affected.</p>
</div>
<div class="paragraph">
<p>One major goal in transpiling Impl projects is to provide the ability to load all modules used in Client/Test projects in non-disruptive way. Even if the implementation is missing elements the runtime should still be able to successfully load the module. Errors should only be signalled when the client code actually uses the missing concepts.</p>
</div>
<div class="sect4">
<h5 id="sec:Modifications_in_Impl_projects"><a class="anchor" href="#sec:Modifications_in_Impl_projects"></a><a class="link" href="#sec:Modifications_in_Impl_projects">12.4.3.1. Modifications in Impl projects</a></h5>
<div class="paragraph">
<p>The generator is module driven. In case of missing modules nothing will be done but the runtime will detect this and act accordingly.</p>
</div>
<div class="paragraph">
<p>In general only missing elements will be inserted:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Missing class - a stub will be generated</p>
</li>
<li>
<p>Missing function - a stub will be generated</p>
</li>
<li>
<p>Missing enumeration - a stub will be generated</p>
</li>
<li>
<p>Missing interface - a stub will be generated</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Missing members of classes are inserted as stubs. Missing fields will be replaced by getter/setter-pairs throwing an error upon read and write access.</p>
</div>
<div class="paragraph">
<p>A more sophisticated approach needs to be taken for interfaces with default implementations (marked with @ProvidesDefaultImplementation or @ProvidesInitializer).</p>
</div>
<div class="paragraph">
<p>Currently missing field initialisers in interfaces are not detected for two reasons: Field-initialising is carried out on loading. Throwing an error in the initialiser will prevent the module from being loaded. Installing a getter/setter pair on the Impl-interface is not an option since the inheritance chain used in client project has no knowledge about this and therefore these accessors cannot be reached from client code.</p>
</div>
<div class="paragraph">
<p>Missing default implementations will be inserted as stubs. For normal class compilation the inheritance chain needs to be scanned. In case of an missing default implementation in an implemented interface a forwarding call to the stub needs to be inserted on the class.</p>
</div>
</div>
<div class="sect4">
<h5 id="sec:Implementation_of_stub_generation"><a class="anchor" href="#sec:Implementation_of_stub_generation"></a><a class="link" href="#sec:Implementation_of_stub_generation">12.4.3.2. Implementation of stub-generation</a></h5>
<div class="paragraph">
<p>The implementation is mainly organised in <code>ApiImplStubGenerationTransformation</code>, which makes use of <code>MissingApiMembersForTranspiler</code> and <code>ScriptApiTracker</code>.</p>
</div>
<div class="paragraph">
<p>When a Module is transpiled the type of the project is checked. Only if the project is an implementation project the comparison between the current module and it’s API module is computed and attached as an Adapter to the <code>Script</code> during the life-cycle of the <code>ScriptTranspilerPolicy</code>. The <code>ProjectComparisonAdapter</code> serves as a shared information pool among the different transpiler-policies and caches different compare-results. After transpilation of the script the adapter will be removed.</p>
</div>
<div class="paragraph">
<p>In order to reuse all existing code as far as possible, missing elements are modelled as subclasses of the requested element but with no AST-information. These subclasses are member-classes of the <code>ScriptApiTracker</code> class. All class-names are prefixed with <code>VirtualApi</code> and hold a reference to the type-information computed by the module-comparison.</p>
</div>
<div class="paragraph">
<p>It is important to not access the AST-elements of type-information obtained from the project-comparison, since this would trigger the AST-loading and invalidate (proxifying) existing <code>EObjects</code>.</p>
</div>
</div>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:n4jsc_Headless_Compiler_Interface"><a class="anchor" href="#sec:n4jsc_Headless_Compiler_Interface"></a><a class="link" href="#sec:n4jsc_Headless_Compiler_Interface">12.5. n4jsc Headless Compiler Interface</a></h3>
<div class="paragraph">
<p>The headless compiler interface consists of a runnable class capable of reading command line options packaged together with all required dependencies into one executable <code>.jar</code> archive.</p>
</div>
<div class="paragraph">
<p>The sources of the command line interface are located in in the <code>tools/</code> subfolder of the main git repository. They comprise of the package <code>org.eclipse.n4js.hlc</code> and corresponding test package <code>org.eclipse.n4js.hlc.tests</code>. (c.f. <a href="#sec:tools">Tools</a>).</p>
</div>
<div class="paragraph">
<p>A description of how to use the headless compiler can be found in the N4IDE document.</p>
</div>
<div class="sect3">
<h4 id="sec:building_the_headless_compiler"><a class="anchor" href="#sec:building_the_headless_compiler"></a><a class="link" href="#sec:building_the_headless_compiler">12.5.1. building the headless compiler</a></h4>
<div class="paragraph">
<p>The maven-modules related to the headless compiler build are organised in the <code>tools</code> folder in the project-root. In order to build the headless compiler as part of the common build-chain, the maven-profile <code>buildTools</code> needs to be activated (off by default), e.g. <code>mvn -PbuildTools</code> .</p>
</div>
<div class="paragraph">
<p>To build the headless compiler separately, the project-pom can be set to <code>tools/pom.xml</code>, however then the tycho-generated artefacts must be accessible by this build. This can be achieved in three different ways:</p>
</div>
<div class="olist loweralpha">
<ol class="loweralpha" type="a">
<li>
<p>the build takes place on a workspace formerly building the n4js-project without cleaning it (interesting for local experiments),</p>
</li>
<li>
<p>a former n4js-build installed the artefacts into the currently configured local-.m2-repository of maven or</p>
</li>
<li>
<p>a former n4js-build deployed the artefacts to the configured nexus-server.</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>Note however, on our build-server option a) is not feasible, option b) requires you to setup a build-chain and ensuring the same build-node to be building on
and c) is difficult up to nearly impossible without a proper versioning-scheme.</p>
</div>
<div class="paragraph">
<p>Parameters for the headless-compiler-build are defined in the parent-pom located at <code>releng/org.eclipse.n4js.parent/pom.xml</code> with properties prefixed by <code>hlc.</code>.</p>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_execution"><a class="anchor" href="#_execution"></a><a class="link" href="#_execution">13. Execution</a></h2>
<div class="sectionbody">
<div class="admonitionblock warning">
<table>
<tr>
<td class="icon">
<i class="fa icon-warning" title=""></i>
</td>
<td class="content">
This chapter may be outdated.
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>There are many different use cases for executing N4JS code:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>running project locally</p>
</li>
<li>
<p>running tests in CI</p>
</li>
<li>
<p>running application in the client</p>
</li>
<li>
<p>running processor on the server</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>All those use cases may differ in their details, but can be divided into general phases:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>execution environment preparation</p>
</li>
<li>
<p>bootstrapping</p>
</li>
<li>
<p>call to given n4js entry point</p>
</li>
<li>
<p>shutdown (optional)</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>When N4JS execution is triggered, proper <em>Runner</em> (see <a href="#sec:Runners-introduction">Runners</a>) is selected. In some cases it is done automatically, in others user needs to make a choice. Runner is responsible for perform all required preparations, according to <a href="#sec:N4JS_Project_Execution_And_Linking_Model">N4JS Project Execution And Linking Model</a>. Then JS execution environment (e.g. NodeJS, IOJS, Chrome, JavaScriptCore) performs bootstrapping according to <a href="#sec:N4JS_Execution_And_Linking_File">N4JS Execution And Linking File</a>. As last step of bootstrap phase defend n4js entry point will be called which starts proper n4js execution phase. In some cases there may be shutdown phase, but that is highly dependent on use case and proceeding execution phases.</p>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:N4JS_Project_Execution_And_Linking_Model"><a class="anchor" href="#sec:N4JS_Project_Execution_And_Linking_Model"></a><a class="link" href="#sec:N4JS_Project_Execution_And_Linking_Model">13.1. N4JS Project Execution And Linking Model</a></h3>
<div class="paragraph">
<p>N4JS project is compiled to JavaScript language, that in turn can be executed in some JS execution environment, Those environments (e.g. NodeJS, IOJS, Chrome, JavaScriptCore) will differ between each other in terms of JS APIs they expose and way JS code has to be provided to them, or the way it is triggered. We introduced systematic way of describing those features in terms of N4JS projects (see Components and Projects <a href="#sec:N4_Components_and_IDE_Support">Components and IDE Support</a>). N4JS project will be of different <em>PojectType</em> that determines project purpose (see Package.json section <a href="#sec:Package_json">Package.json File</a>. When we want to execute some N4JS project, we can divide its dependency graph into 4 general areas</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>User Space, e.g. user code</p>
</li>
<li>
<p>System Space, e.g N4 Platform APIs</p>
</li>
<li>
<p>Runtime Space, e.g. EcmaScript APIs</p>
</li>
<li>
<p>Environment Space, e.g. execution environment APIs</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>Example of that kind of graph can bee seen on <a href="#fig:od_sampleProjectDependencyGraph">Sample Project Dependency Graph</a></p>
</div>
<div id="fig:od_sampleProjectDependencyGraph" class="imageblock center">
<div class="content">
<img src="chapters/14_execution/images/od_sampleProjectDependencyGraph.svg" alt="od sampleProjectDependencyGraph">
</div>
<div class="title">Figure 37. Sample Project Dependency Graph</div>
</div>
<div class="paragraph">
<p>All dependencies are compile time dependency (as they are checked by the compiler), but <em>tend</em> to weaken, the lower in the dependency graph we are. <em>User Space</em> objects will have strong load time and run time dependency to each other and to the <em>System Space</em>. <em>System Space</em> have strong load time and run time dependency to each other and, only runtime dependency to <em>Runtime Space</em>. <em>Runtime Space</em> objects should not have any load time dependencies between each other. In some cases they may have weak runtime dependency to each other. In many cases those components are just api definitions that describe execution environment native apis, but may contain polyfills code. <em>Environment Space</em> has no dependency to other components, except the fact different <em>RuntimeEnvironemnt</em>s can extend each other (see <a href="#sec:N4_Components_and_IDE_Support">[sec:N4_Components_and_IDE_Support]</a>).</p>
</div>
<div class="paragraph">
<p>Runner must configure JS execution environment in the way that all above areas of the dependency graph must be either</p>
</div>
<div class="ulist">
<ul>
<li>
<p>provided by execution environment itself (runtime libraries APIs - <em>n4jsd</em> files)</p>
</li>
<li>
<p>loaded by defined runtime environment (self initialisation code)</p>
</li>
<li>
<p>available to load by environment explicitly (runtime libraries polyfills, system libraries)</p>
</li>
<li>
<p>available to load by other implicitly (system libraries, user libraries and projects)</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Testers, the same way as runners, must be able to execute n4js code. Main difference is that dependency graph for test case will be usually slightly bigger (dependencies to test libraries), and code that has to be triggered shifts a bit from given project to test library used in test code of tested project. Extending previously used example with test elements is shown in figure <a href="#fig:od_sampleTestProjectDependencyGraph2">Sample Test Project Dependency Graph</a>.</p>
</div>
<div id="fig:od_sampleTestProjectDependencyGraph2" class="imageblock center">
<div class="content">
<img src="chapters/14_execution/images/od_sampleTestProjectDependencyGraph.svg" alt="od sampleTestProjectDependencyGraph">
</div>
<div class="title">Figure 38. Sample Test Project Dependency Graph</div>
</div>
<div id="fig:runners-testers" class="imageblock center">
<div class="content">
<img src="chapters/14_execution/images/runners-testers.svg" alt="runners testers">
</div>
<div class="title">Figure 39. Runners and Testers</div>
</div>
<div class="sect3 language-n4js">
<h4 id="subsec:N4JS_Execution_With_NodeJS"><a class="anchor" href="#subsec:N4JS_Execution_With_NodeJS"></a><a class="link" href="#subsec:N4JS_Execution_With_NodeJS">13.1.1. N4JS Execution With NodeJS</a></h4>
<div class="paragraph">
<p>This example shows in-depth details of N4JS code execution with NodeJS runner.</p>
</div>
<div class="paragraph">
<p>In the workspace we have <code>Client</code> with <code>foo.n4js</code> that imports <code>bar.n4js</code> from <code>UserLib</code> that is also in the workspace.
Those N4JS files use some ES5 APIs , e.g. <code>Math.random()</code> and <code>setTimeout()</code>. Those APIs are <code>Global</code> so there is
no impicit import, still they make user projects depend on runtime library <code>n4js-runtime-es2015</code>.
Assuming user selects <code>foo.n4js</code> file for execution, the NodeRunner in the IDE will:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>create working directory in temp folder, e.g. <code>/var/temp/N4JSNodeRun123/</code></p>
</li>
<li>
<p>create <code>node_modules</code> folder inside working directory, to which projects will be linked</p>
</li>
<li>
<p>generate script, e.g. <code>n4jsELF.js</code> that will be responsible for booting execution (see <a href="#sec:N4JS_Execution_And_Linking_File">N4JS Execution And Linking File</a>)</p>
</li>
<li>
<p>runner will put <code>/var/temp/N4JSNodeRun123/node_modules</code> into <code>NODE_PATH</code></p>
</li>
<li>
<p>execute <code>/var/temp/N4JSNodeRun123/n4jsELF.js</code> with NodeJS</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>For example with NodeJS environment if all projects from dependency graph are accessible in local file system, their
paths would need to be put in NodeJS <em>NODE_PATH</em> environment variable. In addition to configuring execution environment
<em>Runner</em> generates N4JS Elf file that is used by environment to bootstrap n4js execution
(see <a href="#sec:N4JS_Execution_And_Linking_File">N4JS Execution And Linking File</a>).
[[fig:od_sampleNodeProjectExecution]</p>
</div>
<div class="imageblock center">
<div class="content">
<img src="chapters/14_execution/images/od_sampleNodeProjectExecution.svg" alt="od sampleNodeProjectExecution">
</div>
<div class="title">Figure 40. Sample NodeJS Project Execution</div>
</div>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:N4JS_Execution_And_Linking_File"><a class="anchor" href="#sec:N4JS_Execution_And_Linking_File"></a><a class="link" href="#sec:N4JS_Execution_And_Linking_File">13.2. N4JS Execution And Linking File</a></h3>
<div class="paragraph">
<p>JS execution environment not only needs to know from where it needs to obtain code to execute, but also <em>what is the entry point to the code that is supposed to be executed</em> and <em>what code needs to be loaded before entry point is called</em>.</p>
</div>
<div class="paragraph">
<p>All this information is generated by the runner based on the executed project dependency graph. The way this information
is presented depends on concrete JS execution environment used, and on its configuration (e.g. user provided options, or
configuration derived in other ways). But in either case file is generated with that information. Figure <a href="#fig:n4js_elf">N4JS ELF examples</a>
shows examples how this information would look like for both testers and runners for both NodeJS or Chrome.</p>
</div>
<div id="fig:n4js_elf" class="imageblock center">
<div class="content">
<img src="chapters/14_execution/images/n4js_elf.svg" alt="n4js elf">
</div>
<div class="title">Figure 41. N4JS ELF examples</div>
</div>
<div class="paragraph">
<p>First segment of the n4js elf is responsible for loading <em>RuntimeEnvironment</em> bootstrap code. Since <em>RuntimeEnvironment</em>s can extend each other, generated information would follow those dependencies. It is possible that <em>RuntimeEnvironment</em>s need to do some special work in regards of of provided <em>RuntimeLibraries</em>s, e.g. initiate initiate polyfills. That code can be either directly in <em>RuntimeEnvironment</em> init code, or its init code can call modules from provided runtime libraries.</p>
</div>
<div class="paragraph">
<p>Second segment of the n4js elf is responsible for loading <em>RuntimeEnvironment</em> exec module. This is special module defined in package.json of the environment, that is used to call into user projects entry point directly, or (as in test case) call into runners of the test library.</p>
</div>
<div class="paragraph">
<p>Last segment of the n4js elf is responsible for passing run/test data (generated by IDE/CLI) into initialised previously exec module.</p>
</div>
<div class="paragraph">
<p>While first two segments are resolved from project dependencies and can be covered by generic approach on IDE/CLI side, last segment requires strong relation between given runner/tester and <em>RuntimeEnvironment</em> / <em>TestEnvironment</em>. While some generic approaches can be used, for the moment we don’t specify concrete convention there.</p>
</div>
<div class="sect3 language-n4js">
<h4 id="subsec:NodeJS_Specific_ELF"><a class="anchor" href="#subsec:NodeJS_Specific_ELF"></a><a class="link" href="#subsec:NodeJS_Specific_ELF">13.2.1. NodeJS Specific ELF</a></h4>
<div class="paragraph">
<p>Concrete environments may need specific setup that is not common for other environemtns. For example for NodeJS runner
needs to configure the node lookup paths for the module resolution. This is achieved by creating at runtime symlinks
from <code>node_modules</code> pointing to concrete dependencies required during execution.</p>
</div>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Runners-execution"><a class="anchor" href="#sec:Runners-execution"></a><a class="link" href="#sec:Runners-execution">13.3. Runners</a></h3>
<div class="paragraph">
<p>It is specified above, that <em>Runner</em> prepares concrete JS execution environment for executing given code and triggers execution process. What is not clear so far is how appropriate runner is selected for given project. In <a href="#_n4components">N4 Components</a> it was specified that N4JS projects do not depend directly on specific runners or JS execution environments. Instead, N4JS tooling should be able to select appropriate runner based on given project transitive dependencies. In this section we specify overall design of runners for both N4JS IDE and CLI tooling and how runners are selected for projects.</p>
</div>
<div class="paragraph">
<p>INFO: In general any n4js code execution is governed by <em>runners</em> and <em>testers</em> depending on the use case.
In this chapter <em>runners</em> are described in detail.
Information from this chapter applies to <em>Testers</em>, unless stated otherwise in chapter dedicated to testing N4JS code (<a href="#_tests">Tests</a>), were we specify
testing specific use cases.</p>
</div>
<div class="sect3">
<h4 id="subsec:N4_Runtime_Environments_Convention"><a class="anchor" href="#subsec:N4_Runtime_Environments_Convention"></a><a class="link" href="#subsec:N4_Runtime_Environments_Convention">13.3.1. N4 Runtime Environments Convention</a></h4>
<div class="paragraph">
<p>Dependency between <em>Runner</em> and <em>Runtime Environment</em> crosses technical boundary between N4JS Projects (N4JS code) and N4JS tooling (IDE and CLI tools implemented with e.g. Java). We introduce convention to implement this dependency, yet letting N4JS projects and N4JS tools internals to be relatively independent.</p>
</div>
<div id="fig:cd_EnvironmentConvention" class="imageblock center">
<div class="content">
<img src="chapters/14_execution/images/cd_EnvironmentConvention.svg" alt="cd EnvironmentConvention">
</div>
<div class="title">Figure 42. Runtime Environments Convention</div>
</div>
<div class="paragraph">
<p><a href="#fig:cd_EnvironmentConvention">Runtime Environments Convention</a> convention that is used to communicate run time configuration of the N4JS projects (grey colour) and N4JSIDE (pink colour). JS projects declare dependencies on provided list of <em>Runtime Libraries</em>. Each combination of those corresponds to one predefined <em>Runtime Environment</em> N4JS component. On N4JSIDE side there is separate list of <em>runtime environment</em>s maintained. Both lists correspond one to one to each other.</p>
</div>
</div>
<div class="sect3 language-n4js">
<h4 id="subsec:Passing_Information_from_IDE_to_Execution_Code_in_Runtime_Environment"><a class="anchor" href="#subsec:Passing_Information_from_IDE_to_Execution_Code_in_Runtime_Environment"></a><a class="link" href="#subsec:Passing_Information_from_IDE_to_Execution_Code_in_Runtime_Environment">13.3.2. Passing Information from IDE to Execution Code in Runtime Environment</a></h4>
<div class="paragraph">
<p>When launching an N4JS file, the IDE will compute some information on the containing N4JS project and its direct and indirect dependencies as well as the runtime environment in use. This information will be passed on to the execution module defined in the runtime environment, i.e. the code specified via property in the runtime environment’s package.json file. The information will be passed via a global variable <code>$executionData</code>. The value will be a Javascript object with the following properties:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>userSelection</code>: the module the user had selected when initiating the launch. This will usually be the <code>module to run</code> , but in case of testing it will be the project, folder, or file the user had selected.</p>
</li>
<li>
<p><code>projectNameMapping</code>: an object in which every key is the name of an API project among the direct or indirect dependencies of the project being run, and every value is the name of the corresponding implementation project being used. When running N4JS projects that do not make use of the API / implementation project technique, then this property will either hold an empty object or be undefined.</p>
</li>
<li>
<p><code>testTree</code> (only when running tests): the test tree as defined in <a href="#_tests">Tests</a> containing information on the tests to be run, i.e. test classes, test methods, etc. The test tree will be encoded as JSON, so the value of this property will be of type string and should be passed to <code>JSON.parse()</code>.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>All calculations described above are based on the workspace available. This includes library manager functionality, see <a href="../20_externalLibraries/externalLibraries.html#:External_Library_Workspace">External Library Workspace</a>.
In specific setups, where workspace is not available runners provide helper utility <code>org.eclipse.n4js.runner.RunnerFileBasedShippedCodeConfigurationHelper.configureFromFileSystem()</code> that allows to configure given <code>RunConfiguration</code> using
plain file system to the external libraries. Note that in order to do this in a way that allows to re-use all computation logic based on <a href="../12_n4components/n4components.html#:N4MFContainerManagement">N4Containers</a>,
runners infrastructure provides its own subclasses of the few component types. Those specialized types are used only in scope of <code>RunnerFileBasedShippedCodeConfigurationHelper</code> and are not exposed to the rest of the system.</p>
</div>
<div class="paragraph">
<p>Specific runners, e.g. the NodeJS or Chrome runner, may choose to provide more information via the execution data object.</p>
</div>
</div>
<div class="sect3">
<h4 id="subsec:Runners_Design"><a class="anchor" href="#subsec:Runners_Design"></a><a class="link" href="#subsec:Runners_Design">13.3.3. Runners Design</a></h4>
<div class="paragraph">
<p>As specified in section before N4JS projects will need to be executed on various JS execution environments, for which dedicated runners will be needed. While they will differ how they interact with concrete JS environment, they will have common parts when it comes to interaction with N4JS IDE or CLI. Those parts are provided in form of abstract <em>IDERunner</em> <em>CLIRunner</em> and <em>Runner</em> <em>components</em> (or bundles) that specific runners should use to interact with N4JS IDE or CLI.</p>
</div>
<div class="paragraph">
<p>Runner by design consists of three parts:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><em>core</em> part (green colour) - contains most logic, resources (e.g. JS execution environment binary)</p>
</li>
<li>
<p><em>IDE</em> part (blue colour) - responsible for working with N4JS IDE (enabling runner in ui, providing views)</p>
</li>
<li>
<p><em>CLI</em> part (yellow colour) - responsible for working with N4JS CLI (get command line parameters, provide console output)</p>
</li>
</ul>
</div>
<div id="cd_RunnersIdeCli" class="imageblock center">
<div class="content">
<img src="chapters/14_execution/images/cd_RunnersIdeCli.svg" alt="cd RunnersIdeCli">
</div>
<div class="title">Figure 43. Runner for N4JS IDE and CLI tooling</div>
</div>
<div class="paragraph">
<p>Specific runner is connected to running N4JS IDE or CLI via extension points. This is done either by using them directly, or by using types exposed by abstract runner component.</p>
</div>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Legacy_Execution_Engine"><a class="anchor" href="#sec:Legacy_Execution_Engine"></a><a class="link" href="#sec:Legacy_Execution_Engine">13.4. Legacy Execution Engine</a></h3>
<div class="paragraph">
<p>Compilation of N4JS may target many platforms. For the moment it is hard to discuss what they will be exactly or if N4JSIDE will provide some integration or hooks to those platforms. On the other hand we want to have some execution environment for internal use to validate behaviour of compiled code. Since we know that V8 based platforms (e.g. Chrome, NodeJS) will be in our target platforms set we want to be able to execute compiled code on similar environment. As standalone V8 integration is quite challenging, we have decided to integrate in N4JSIDE NodeJS as execution environment. This is considered internal feature used for testing compilation of N4JS and N4JSIDE.</p>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Design"><a class="anchor" href="#sec:Design"></a><a class="link" href="#sec:Design">13.5. Design</a></h3>
<div class="paragraph">
<p>We provide NodeJS binaries for various OSes. Direct access to binaries is not exposed. Selection and institutionalization of the binary is done internally and is not configurable. Instead bundle containing binaries provides classes required to run code (in form of <code>String</code> or <code>File</code> with the code). Clients That want to do this may either use provided <code>Engine</code> class or can implement their own engine based on provided infrastructure. Main class that used in engine implementation is <code>EngineCommandBuilder</code>. This class is responsible for building proper command line commands that engine implementation must execute to run code with NodeJS.</p>
</div>
<div class="paragraph">
<p><a href="#fig:cd_executionengine">NodeJS execution integration</a> shows the most important classes of the NodeJS integration.</p>
</div>
<div id="fig:cd_executionengine" class="imageblock center">
<div class="content">
<img src="chapters/14_execution/images/cd_executionengine.svg" alt="cd executionengine">
</div>
<div class="title">Figure 44. NodeJS execution integration</div>
</div>
<div class="sect3">
<h4 id="sec:Usage_Outside_N4JSIDE"><a class="anchor" href="#sec:Usage_Outside_N4JSIDE"></a><a class="link" href="#sec:Usage_Outside_N4JSIDE">13.5.1. Usage Outside N4JSIDE</a></h4>
<div class="paragraph">
<p>In this use case we use provdied <code>Engine</code> class that allows to execute js code in form of <code>String</code> or <code>File</code> with the code. In return user receives <code>EngineOutput</code> object with two lists of strings containing standard output and error output of the node process, that were captured during execution.</p>
</div>
<div class="paragraph">
<p>In this usage scenario execution api assumes valid JS code. User needs to ensure compilation of code prior to execution, if needed.</p>
</div>
<div class="paragraph">
<p>That functionality is used in internal jUnit tests and in xpect tests of the compiler.</p>
</div>
<div class="sect4">
<h5 id="sec:Use_Node_with_Maven"><a class="anchor" href="#sec:Use_Node_with_Maven"></a><a class="link" href="#sec:Use_Node_with_Maven">13.5.1.1. Use Node with Maven</a></h5>
<div class="paragraph">
<p>Note on maven usage. For maven based builds we need to ensure that binary resources are available and are unpacked. To do this in pom of the project that will be calling engine we must include following listing:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-xml" data-lang="xml"> &lt;plugin&gt;
&lt;groupId&gt;org.apache.maven.plugins&lt;/groupId&gt;
&lt;artifactId&gt;maven-dependency-plugin&lt;/artifactId&gt;
&lt;executions&gt;
&lt;execution&gt;
&lt;id&gt;unpack&lt;/id&gt;
&lt;phase&gt;process-test-classes&lt;/phase&gt;
&lt;goals&gt;
&lt;goal&gt;unpack&lt;/goal&gt;
&lt;/goals&gt;
&lt;configuration&gt;
&lt;artifactItems&gt;
&lt;artifactItem&gt;
&lt;groupId&gt;org.eclipse.n4js&lt;/groupId&gt;
&lt;artifactId&gt;org.eclipse.n4js.js.engine&lt;/artifactId&gt;
&lt;version&gt;0.0.1-SNAPSHOT&lt;/version&gt;
&lt;overWrite&gt;true&lt;/overWrite&gt;
&lt;outputDirectory&gt;
${project.build.directory}/classes
&lt;/outputDirectory&gt;
&lt;/artifactItem&gt;
&lt;/artifactItems&gt;
&lt;/configuration&gt;
&lt;/execution&gt;
&lt;/executions&gt;
&lt;/plugin&gt;</code></pre>
</div>
</div>
</div>
</div>
<div class="sect3">
<h4 id="sec:Usage_Inside_N4JSIDE"><a class="anchor" href="#sec:Usage_Inside_N4JSIDE"></a><a class="link" href="#sec:Usage_Inside_N4JSIDE">13.5.2. Usage Inside N4JSIDE</a></h4>
<div class="paragraph">
<p>In Eclipse platfrom based environment we use custom implementation of the engine, <code>PlatformEngine</code>. This implementation unlike default <code>Platform</code> is non blocking implementation that forwards output to platform console and allows platform to control lifecycle of the running engine. Additionally this version uses its own implementation of the <code>ResourceUrlResolver</code>. It is required to properly resolve urls that point inside platform bundles.</p>
</div>
<div class="paragraph">
<p>This scenario we assume that user will run N4JS files or JS files. We have created proper UI hooks that allow user to do this either from editor or as selection menu. Based on name of the file that user commands to execute we find proper compiled file (compiled with our ES5 compiler - it is not configurable). When found we execute this file in our execution engine. If it is not found, execution engine will write appropriate error to N4JSIDE console.</p>
</div>
</div>
</div>
<div class="sect2 langauge-n4js">
<h3 id="sec:Runtime_Injection"><a class="anchor" href="#sec:Runtime_Injection"></a><a class="link" href="#sec:Runtime_Injection">13.6. Runtime Injection</a></h3>
<div class="paragraph">
<p>There is need to inject into runtime environment some special code, for example when compiling N4JS to ES5 (see N4JavaScriptSpecification,chapter N4 JS Compilation). To achieve this wee need to inject desired code when calling engine to run desired compiled code. Injection mechanism depends a lot on way we run engine. In this section injection of runtime is discussed based on NodeJs that is used as runtime environment.</p>
</div>
<div class="sect3">
<h4 id="sec:Running_String_Code"><a class="anchor" href="#sec:Running_String_Code"></a><a class="link" href="#sec:Running_String_Code">13.6.1. Running String Code</a></h4>
<div class="paragraph">
<p>We allow code execution where code is provided in form of <code>String</code>. In this case we are calling nodejs with parameters . To enrich execution environment in this case we are appending special runtime code at the end of file. It is important to append it at the end, to avoid changing line numbers of original code and decrease other potential side effects. So actual invocation of nodejs looks like</p>
</div>
<div class="paragraph">
<p>This mechanism assumes:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>injected code starts with new line - this makes ASI mechanism to finish user last statement if it was not properly finished by user, otherwise just creates</p>
</li>
<li>
<p>injected code does not have to be initiated manually - all exposed api is in named function declarations</p>
</li>
</ol>
</div>
<div class="paragraph">
<p><em>Explanation</em></p>
</div>
<div class="paragraph">
<p>In first assumption we make workaround for user code that does not contain new line or semicolon at the end of last statement. This kind of code is incorrect and would result in last statement of user code and first statement of injected code to be interpreted as one JS statement. In most cases that would be invalid code. By having new line as first character of injected code, we are taking advantage of JS AutomaticSemicolonInjection mechanism. If user code AST is not finished properly this mechanism will finish close user AST. If user AST is finished properly, ASI will just insert empty statement between user code and injection code. In both cases we end up with proper AST.</p>
</div>
<div class="paragraph">
<p>Second assumption avoids need for further user code modifications, as injected does not have to be manually called. Instead we take advantage of variable and function hoisting mechanism of JS. This assures that even though user code is first in AST, JS environment will first initiate named functions therefore when user code calls injected code it is already defined in scope in which user code executes.</p>
</div>
</div>
<div class="sect3">
<h4 id="sec:Running_File_Code"><a class="anchor" href="#sec:Running_File_Code"></a><a class="link" href="#sec:Running_File_Code">13.6.2. Running File Code</a></h4>
<div class="paragraph">
<p>Second method of code execution is to execute provided file with user code. Normal way of doing that with NodeJS is to make call. But since we need to inject special code without rewriting files, we use different mechanism. Basically we are executing injected code and in the same scope using node api. Additionally we are attaching injected code to global scope in node, ensuring this way that required file is executed in scope which contains injected code. Putting this all together we are making following call:</p>
</div>
<div class="paragraph">
<p>This mechanism assumes that injected code attaches all exposed API to global scope .</p>
</div>
</div>
<div class="sect3">
<h4 id="sec:Injection_Code_Example"><a class="anchor" href="#sec:Injection_Code_Example"></a><a class="link" href="#sec:Injection_Code_Example">13.6.3. Injection Code Example</a></h4>
<div class="paragraph">
<p>Following is simple example of properly formed injection code.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-n4js" data-lang="n4js">;
function foo(){}
function bar(){}
function baz(){}
(function(){
GLOBAL.foo = foo;
GLOBAL.bar = bar;
GLOBAL.baz = baz;
})();
;</code></pre>
</div>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>first line is empty line to trigger ASI</p>
</li>
<li>
<p>second line (optional) enters</p>
</li>
<li>
<p>lines 3-5 are defining runtime api in current scope (in which user code provided as a string is executed)</p>
</li>
<li>
<p>lines 6 (optional) is just a visual sugar</p>
</li>
<li>
<p>lines 7-11 are adding runtime api to global scope (to expose it when runnig user file with code)</p>
</li>
<li>
<p>lines 12-13 (optional) are there to separate injected code and invokation of user file (if running user provided file with code)</p>
</li>
</ol>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_tests"><a class="anchor" href="#_tests"></a><a class="link" href="#_tests">14. Tests</a></h2>
<div class="sectionbody">
<div class="admonitionblock warning">
<table>
<tr>
<td class="icon">
<i class="fa icon-warning" title=""></i>
</td>
<td class="content">
This chapter may be outdated.
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>In order to run all tests from command line, use maven:</p>
</div>
<div class="listingblock">
<div class="content">
<pre>mvn clean verify</pre>
</div>
</div>
<div class="paragraph">
<p>You may have to increase the memory for maven via <code>export MAVEN_OPTS="-Xmx2048m"</code> (Unix) or <code>set MAVEN_OPTS="-Xmx2048m"</code> (Windows).</p>
</div>
<div class="paragraph">
<p>Do not run the tests via <code>mvn clean test</code> as this may lead to some failures.</p>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Performance_Tests"><a class="anchor" href="#sec:Performance_Tests"></a><a class="link" href="#sec:Performance_Tests">14.1. Performance Tests</a></h3>
<div class="paragraph">
<p>There are two kind of performance tests:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Synthetic Tests: an arbitrary number of test classes is generated, and then some modifications are performed on these classes.</p>
</li>
<li>
<p>Real World Tests: tests are based on a snapshot version of our platform libraries</p>
</li>
</ol>
</div>
<div class="sect3">
<h4 id="sec:Synthetic_Performance_Tests"><a class="anchor" href="#sec:Synthetic_Performance_Tests"></a><a class="link" href="#sec:Synthetic_Performance_Tests">14.1.1. Synthetic Performance Tests</a></h4>
<div class="paragraph">
<p>The idea of the synthetic performance tests is to test the performance of specific functionality with a defined number classes, specially designed for the functionality under test.</p>
</div>
<div class="paragraph">
<p>The overall structure of the synthetic performance test is</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>generate test classes</p>
</li>
<li>
<p>compile these classes</p>
</li>
<li>
<p>modify the test classes</p>
</li>
<li>
<p>measure incremental build time</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>Step 3) and 4) can be done in a loop. Also, step 2) can be looped (with clean build).</p>
</div>
<div class="paragraph">
<p>The test classes are spread over clusters and projects. The following categories are used:</p>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Cluster
</td>
<td class="hdlist2">
<p>A cluster is a set of projects, each project of a cluster may depend on another project of the cluster. There are no dependencies between projects of different clusters</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Project
</td>
<td class="hdlist2">
<p>A project simply is a N4JS project, containing packages. A project may depend on other projects.</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Package
</td>
<td class="hdlist2">
<p>A package is a folder in a source folder of a project. A package contains classes.</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Class
</td>
<td class="hdlist2">
<p>A class is defined in a file, usually one class per file. The file, and with it the class, is contained in a package. The class contains members.</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Member
</td>
<td class="hdlist2">
<p>A member is either a field or method of a class. A method may has a body, which may contain variables with references to other classes.</p>
</td>
</tr>
</table>
</div>
<div class="sect4 language-n4js">
<h5 id="sec:Design_of_Generator"><a class="anchor" href="#sec:Design_of_Generator"></a><a class="link" href="#sec:Design_of_Generator">14.1.1.1. Design of Generator</a></h5>
<div class="paragraph">
<p><a href="#fig:cd_performancetest_generator">Performance Generator</a> shows the classes of the performance test generator.</p>
</div>
<div id="fig:cd_performancetest_generator" class="imageblock">
<div class="content">
<img src="chapters/15_tests/images/cd_performancetest_generator.svg" alt="cd performancetest generator">
</div>
</div>
<div class="paragraph">
<p>The package is designed as follows:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p><code>N4ProjectGenerator</code> main control class for generation</p>
</li>
<li>
<p><code>TestDescriptor</code> and subclasses: In order to <em>keep memory consumption of the test class generator low</em>, there is no graph structure created for the test elements. Instead, each element is uniquely named by a number, this number (actually a tuple of numbers) is stored in <code>TestDescriptors</code> and sub classes. There is a descriptor for each element of the tests.</p>
</li>
<li>
<p><code>AbstractModifier</code> and subclasses generarate the tests. The idea is as follows:</p>
<div class="ulist">
<ul>
<li>
<p><code>Modifier</code> generates all files, with complete references and no issues (complete)</p>
</li>
<li>
<p>sub classes of <code>Modifier</code> skip certain generations or add modifications, leading to issues or solving them</p>
</li>
</ul>
</div>
</li>
</ol>
</div>
<div class="paragraph">
<p>In order to compute the name of a class from its descriptor, as well as retrieving a class based on an absolute number, the modifiers use utility methods provided by <code>PerformanceTestConfiguration</code>. Note that computing the names and numbers depends on a configuration!</p>
</div>
</div>
<div class="sect4">
<h5 id="sec:Design_of_Performance_Test_Execution"><a class="anchor" href="#sec:Design_of_Performance_Test_Execution"></a><a class="link" href="#sec:Design_of_Performance_Test_Execution">14.1.1.2. Design of Performance Test Configuration and Execution</a></h5>
<div class="paragraph">
<p><a href="#fig:cd_performancetest_configAnRun">Class Diagram of Performance Test Configuration and Execution</a> shows the classes of the performance test configuration and execution.</p>
</div>
<div id="fig:cd_performancetest_configAnRun" class="imageblock">
<div class="content">
<img src="chapters/15_tests/images/cd_performancetest_configAnRun.svg" alt="cd performancetest configAnRun">
</div>
<div class="title">Figure 45. Class Diagram of Performance Test Configuration and Execution</div>
</div>
<div class="paragraph">
<p>The package is designed as follows:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p><code>PerformanceTestConfiguration</code> stores the test configuration. The configuration stores how many clusters, packages etc. are to be generated. It also provides methods for generating names from the descriptors mentioned above.</p>
</li>
<li>
<p><code>PerformanceMeter</code> executes the test, listening to the (build) job to be finished etc.</p>
</li>
<li>
<p><code>AbstractGeneratingPerformanceTest</code> Base test class contains setup, teardown and utility methods.</p>
</li>
<li>
<p><code>PerformanceTest</code> Test class containing tests.</p>
</li>
</ol>
</div>
</div>
<div class="sect4">
<h5 id="sec:JUnit_Configuration"><a class="anchor" href="#sec:JUnit_Configuration"></a><a class="link" href="#sec:JUnit_Configuration">14.1.1.3. JUnit Configuration</a></h5>
<div class="paragraph">
<p>We are using JUnitBenchamrks (<a href="http://labs.carrotsearch.com/junit-benchmarks.html/" class="bare">http://labs.carrotsearch.com/junit-benchmarks.html/</a>) to extend adjust plain JUnit behavior specifically to the performance tests needs.</p>
</div>
</div>
<div class="sect4">
<h5 id="sec:JUnitBenchmark_Test_Configuration"><a class="anchor" href="#sec:JUnitBenchmark_Test_Configuration"></a><a class="link" href="#sec:JUnitBenchmark_Test_Configuration">14.1.1.4. JUnitBenchmark Test Configuration</a></h5>
<div class="paragraph">
<p>JUnitBenchmark test configuration performed by annotating test method with <code>@BenchmarkOptions</code>. Parameters for that annotation include:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p><code>warmupRounds</code> how many times test will be executed without taking measurement</p>
</li>
<li>
<p><code>benchmarkRounds</code> how many times test will be executed, measurements taken will be used in results report</p>
</li>
<li>
<p><code>callgc</code> Call <code>System.gc()</code> before each test. This may slow down the tests in a significant way.</p>
</li>
<li>
<p><code>concurrency</code> specifies how many threads to use for tests.</p>
</li>
<li>
<p><code>clock</code> specifies which clock to use.</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>Typical configuration for our performance tests might look like:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code> @BenchmarkOptions(benchmarkRounds = 5, warmupRounds = 2)
@Test
public void test() throws Exception {
//test...
}</code></pre>
</div>
</div>
</div>
<div class="sect4">
<h5 id="sec:JUnitBenchmark_Report_Configuration"><a class="anchor" href="#sec:JUnitBenchmark_Report_Configuration"></a><a class="link" href="#sec:JUnitBenchmark_Report_Configuration">14.1.1.5. JUnitBenchmark Report Configuration</a></h5>
<div class="paragraph">
<p>By annotating TestClass in proper way, JUnitBenchamrks will generate html reports with performance results. There are two reports that can be generated:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p><code>@BenchmarkMethodChart</code> report will contain results for every method from one test run (but all <code>benchmarkRounds</code> defined)</p>
<div class="ulist">
<ul>
<li>
<p><code>filePrefix</code> defines report file name</p>
</li>
</ul>
</div>
</li>
<li>
<p><code>@BenchmarkHistoryChart</code> report will contain trend of results from multiple test runs (it is aggregation of multiple instances of <code>@BenchmarkMethodChart</code> report)</p>
<div class="ulist">
<ul>
<li>
<p><code>filePrefix</code> defines report file name</p>
</li>
<li>
<p><code>labelWith</code> defines label that will mark separate runs</p>
</li>
</ul>
</div>
</li>
</ol>
</div>
<div class="paragraph">
<p><em>labelWith</em> property can have value propagated from run configuration/command line. example configuration might be <code>@BenchmarkHistoryChart(filePrefix = benchmark-history, labelWith = LabelType.CUSTOM_KEY)</code></p>
</div>
</div>
<div class="sect4">
<h5 id="sec:JUnitBenchmark_Run_Configuration"><a class="anchor" href="#sec:JUnitBenchmark_Run_Configuration"></a><a class="link" href="#sec:JUnitBenchmark_Run_Configuration">14.1.1.6. JUnitBenchmark Run Configuration</a></h5>
<div class="paragraph">
<p>It is possible to specify additional options for performance test run</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p><code>-Djub.consumers=CONSOLE,H2</code> specifies where results will be written, <em>H2</em> indicates H2 database to be used</p>
</li>
<li>
<p><code>-Djub.db.file=.benchmarks</code> specifies name of the H2 database file</p>
</li>
<li>
<p><code>-Djub.customkey=</code> value of that property scan be used as label in <code>@BenchmarkHistoryChart</code></p>
</li>
</ol>
</div>
</div>
<div class="sect4">
<h5 id="sec:JUnitBenchmark_Example"><a class="anchor" href="#sec:JUnitBenchmark_Example"></a><a class="link" href="#sec:JUnitBenchmark_Example">14.1.1.7. JUnitBenchmark Example</a></h5>
<div class="paragraph">
<p>configuration example:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>@BenchmarkMethodChart(filePrefix = "benchmark-method")
@BenchmarkHistoryChart(filePrefix = "benchmark-history", labelWith = LabelType.CUSTOM_KEY)
public class PerformanceTest extends AbstractGeneratingPerformanceTest {
public PerformanceTest() {
super("PerfTest");
}
@Rule
public TestRule benchmarkRun = new BenchmarkRule();
@Test
@BenchmarkOptions(benchmarkRounds = 5, warmupRounds = 2)
public void Test1() throws Exception {
//Test...
}
@Test
@BenchmarkOptions(benchmarkRounds = 5, warmupRounds = 2)
public void Test2() throws Exception {
//Test...
}
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>executing this code in Eclipse with configuration:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-bash" data-lang="bash">-Xms512m -Xmx1024m -XX:MaxPermSize=512m $-$Djub.consumers=CONSOLE,H2 $-$Djub.db.file=.benchmarks $-$Djub.customkey=${current_date}</code></pre>
</div>
</div>
<div class="paragraph">
<p>will cause :</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>both tests to be executed 2 times for the warmup</p>
</li>
<li>
<p>both of tests being executed 5 times with measurement taken</p>
</li>
<li>
<p>results written to console</p>
</li>
<li>
<p>results stored in local H2 db file (created if doesn’t exist)</p>
</li>
<li>
<p>generated <em>benchmark-method.html</em> with performance results of every test in that execution</p>
</li>
<li>
<p>generated <em>benchmark-history.html</em> with performance results of every execution</p>
</li>
<li>
<p>separate test executions will be labeled in <em>benchmark-history.html</em> with their start time</p>
</li>
</ol>
</div>
</div>
<div class="sect4">
<h5 id="sec:Note_on_Jenkins_Job"><a class="anchor" href="#sec:Note_on_Jenkins_Job"></a><a class="link" href="#sec:Note_on_Jenkins_Job">14.1.1.8. Note on Jenkins Job</a></h5>
<div class="paragraph">
<p>For performance tests it is important not to get pass/fail result in terms of being below given threshold, but also to examine trend of those results. We achieve this by tooling described above. In order to keep this data independent of the build machine or build system storage, we are using separate repository to store performance artifacts. Jenkins in copying previous test results into workspace, runs performance tests, then commits and pushes combined results (adds current results to previous results) to repository.</p>
</div>
</div>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:ECMA_Tests"><a class="anchor" href="#sec:ECMA_Tests"></a><a class="link" href="#sec:ECMA_Tests">14.2. ECMA Tests</a></h3>
<div class="paragraph">
<p>ECMAScript Language test262 is a test suite intended to check agreement between JavaScript implementations and ECMA-262, the ECMAScript Language Specification (currently 5.1 Edition).The test suite contains thousands of individual tests, each of which tests some specific requirements of the ECMAScript Language Specification. For more info refer to <a href="http://test262.ecmascript.org/" class="bare">http://test262.ecmascript.org/</a></p>
</div>
<div class="paragraph">
<p>Uses of this suite may include:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>grammar tests</p>
</li>
<li>
<p>validation tests</p>
</li>
<li>
<p>run-time tests</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>ECMA test262 suite source code can be found here: <a href="http://hg.ecmascript.org/tests/test262" class="bare">http://hg.ecmascript.org/tests/test262</a></p>
</div>
<div class="sect3">
<h4 id="sec:Grammar_Tests"><a class="anchor" href="#sec:Grammar_Tests"></a><a class="link" href="#sec:Grammar_Tests">14.2.1. Grammar Tests</a></h4>
<div class="paragraph">
<p>Based on the JS files included in test262 suite we are generating tests that feed provided JS code into the parser. This operation will result in</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>parser throwing exceptions</p>
</li>
<li>
<p>parsed output will contain standard output</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>First case indicates that parsing provided JS code was not possible. This is considered to be Test Error.</p>
</div>
<div class="paragraph">
<p>Second case case indicates that parsing of the provided code was successful, and will result either</p>
</div>
<div class="ulist">
<ul>
<li>
<p>output with no errors - code adhered parser grammar</p>
</li>
<li>
<p>output with errors - code violated parser grammar</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Given test must interpret those results to provide proper test output.</p>
</div>
<div class="sect4">
<h5 id="sec:Negative_Tests"><a class="anchor" href="#sec:Negative_Tests"></a><a class="link" href="#sec:Negative_Tests">14.2.1.1. Negative Tests</a></h5>
<div class="paragraph">
<p>It is important to note that some of the tests are positive and some are negative. Negative test cases are marked by the authors with <em>@negative</em> JSDoc like marker therefore parser tests must be aware of that to avoid both false positives and false negatives results.</p>
</div>
</div>
<div class="sect4">
<h5 id="sec:Test_Exclusion"><a class="anchor" href="#sec:Test_Exclusion"></a><a class="link" href="#sec:Test_Exclusion">14.2.1.2. Test Exclusion</a></h5>
<div class="paragraph">
<p>To exclude validation tests or run-time related test, implementation is blacklist approach to exclude some of the ECMA test262 tests from execution.</p>
</div>
</div>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Integration_Tests"><a class="anchor" href="#sec:Integration_Tests"></a><a class="link" href="#sec:Integration_Tests">14.3. Integration Tests</a></h3>
<div class="paragraph">
<p>Integration tests based on the stdlib and online-presence code bases can be found in bundle <code>org.eclipse.n4js.hlc.tests</code>
in package <code>org.eclipse.n4js.hlc.tests.integration</code> (headless case) and in bundle <code>org.eclipse.n4js.ui.tests</code> in
package <code>org.eclipse.n4js.ui.tests.integration</code> (plugin-UI tests running inside Eclipse). The headless
tests also execute mangelhaft tests, the UI tests only perform compilation of the test code.</p>
</div>
<div class="paragraph">
<p>More information can be found in the API documentation of classes <code>AbstractIntegrationTest</code> and <code>AbstractIntegrationPluginUITest</code>.</p>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Test_Helpers"><a class="anchor" href="#sec:Test_Helpers"></a><a class="link" href="#sec:Test_Helpers">14.4. Test Helpers</a></h3>
<div class="paragraph">
<p>Test helpers contain utility classes that are reused between different test plug-ins.</p>
</div>
<div class="sect3">
<h4 id="sec:Parameterized_N4JS_Tests"><a class="anchor" href="#sec:Parameterized_N4JS_Tests"></a><a class="link" href="#sec:Parameterized_N4JS_Tests">14.4.1. Parameterized N4JS tests</a></h4>
<div class="paragraph">
<p>Xtext JUnit test runer injects test a ParserHelper that allows to run N4JS parser on given input and obtain information abut parsing results. In some cases we want to run this kind of tests on large input data. To address this we provide two utilities ParameterizedXtextRunner and TestCodeProvider. They allow write data driven parser tests.</p>
</div>
<div class="sect4">
<h5 id="sec:ParameterizedXtextRunner"><a class="anchor" href="#sec:ParameterizedXtextRunner"></a><a class="link" href="#sec:ParameterizedXtextRunner">14.4.1.1. ParameterizedXtextRunner</a></h5>
<div class="paragraph">
<p>This This junit runner serves two purposes:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>injecting ParserHelper</p>
</li>
<li>
<p>creating multiple test instances for each input data provided</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>This class is based on @link org.eclipse.xtext.testing.XtextRunner and @link org.junit.runners.Parameterized</p>
</div>
</div>
<div class="sect4">
<h5 id="sec:TestCodeProvider"><a class="anchor" href="#sec:TestCodeProvider"></a><a class="link" href="#sec:TestCodeProvider">14.4.1.2. TestCodeProvider</a></h5>
<div class="paragraph">
<p>This class is repsonsible for extracting ZipEntry from provided ZipFile. Additinally it can filter out entries that match strings in provided black list file. Filtering out ZipEntries assumes that blacklist file contians Path of ZipEntry in ZipFile as string in one line. Lines starting with <em>#</em> in black list file are ignored by TestCodeProvider.</p>
</div>
</div>
<div class="sect4">
<h5 id="sec:Example_Of_Parameterized_Parser_Test"><a class="anchor" href="#sec:Example_Of_Parameterized_Parser_Test"></a><a class="link" href="#sec:Example_Of_Parameterized_Parser_Test">14.4.1.3. Example of parameterized parser test</a></h5>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-n4js" data-lang="n4js">@RunWith(XtextParameterizedRunner.class)
@InjectWith(N4JSInjectorProvider.class)
public class DataDrivenParserTestSuite {
/**
* Zip archives containing test files.
*/
public static final Collection&lt;String&gt; TEST_DATA_RESOURCES = Arrays.asList("foo.zip", "bar.zip");
/**
* Blacklist of files requiring an execution engine.
*/
public static final String BLACKLIST_FILENAME = "blacklist.txt";
/**
* Every generated test will use different ZipEntry as test data
*/
final ZipEntry entry;
/**
* Name of resource containing corresponding ZipEntry
*/
final String resourceName;
@Inject
protected ParseHelper&lt;Script&gt; parserN4JS;
Collection&lt;String&gt; blackList;
static final Logger logger = Logger.getLogger("someLogger");
public CopyOfLibraryParsingTestSuite(ZipEntry entry, String resourceName, Collection&lt;String&gt; blackList) {
this.entry = entry;
this.resourceName = resourceName;
this.blackList = blackList;
}
@Rule
public TestRule blackListHandler = new TestRule() {
@Override
public Statement apply(final Statement base, Description description) {
final String entryName = entry.getName();
if (blackList.contains(entryName)) {
return new Statement() {
@Override
public void evaluate() throws Throwable {
try {
base.evaluate();
} catch (AssertionError e) {
// expected
return;
}
}
};
} else {
return base;
}
}
};
/**
* Generates collection of ZipEntry instances that will be used as data
* provided parameter is mapped to name of the test (takes advantage of fact
* that ZipEntry.toString() is the same as entry.getName())
*
* @return
* @throws URISyntaxException
* @throws ZipException
* @throws IOException
*/
@Parameters(name = "{0}")
public static Collection&lt;Object[]&gt; data() throws URISyntaxException, ZipException, IOException {
return TestCodeProvider.getDataFromZippedRoots(TEST_DATA_RESOURCES, BLACKLIST_FILENAME);
}
/**
* generated instances of the tests will use this base implementation
*
* @throws Exception
*/
@Test
public void test() throws Exception {
assertNotNull(this.entry);
assertNotNull(this.resourceName);
assertNotNull(this.parserN4JS);
//actual test code
}
}</code></pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Issue_Suppression"><a class="anchor" href="#sec:Issue_Suppression"></a><a class="link" href="#sec:Issue_Suppression">14.5. Issue Suppression</a></h3>
<div class="paragraph">
<p>It can be useful to suppress certain issues before tests are ran, so that test expectations don’t have to consider inessential warnings. This means that the validator still returns a full list of issues, but before passing them to the testing logic, the issues are filtered.</p>
</div>
<div class="paragraph">
<p>When working with JUnit tests, the custom InjectorProvider <code>N4JSInjectorProviderWithIssueSuppression</code> can be used to configure them to suppress issues.</p>
</div>
<div class="paragraph">
<p>The codes that are suppressed are globally specified by the<br>
<code>DEFAULT_SUPPRESSED_ISSUE_CODES_FOR_TESTS</code> constant in <code>N4JSLanguageConstants</code>.</p>
</div>
<div class="paragraph">
<p>When working with Xpect tests, the XpectSetupFactory <code>SuppressIssuesSetup</code> can be used. See <a href="#sec:Xpect_Issue_Suppression">Xpext Issue Suppression</a> for more details on Xpect issue suppression.</p>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Xpect_Tests"><a class="anchor" href="#sec:Xpect_Tests"></a><a class="link" href="#sec:Xpect_Tests">14.6. Xpect Tests</a></h3>
<div class="paragraph">
<p>For many tests, Xpect [<a href="#Xpect">Xpect</a>] is used. Xpect allows for defining tests inside the language which is the language under test. That is, it is possible to refer to a JUnit test method in a special annotated comment, along with arguments passed to that method (typically expectations and the concrete location). Xpect comes with a couple of predefined methods which could be used there, e.g., tests for checking whether some expected error messages actually are produced. We have defined (and will probably define more) N4JS specific test methods.</p>
</div>
<div class="paragraph">
<p>In the following, we describe the most common Xpect test methods we use. Note that we do not use all types of tests shipped with Xpect. For example, AST tests (comparing the actual AST with an expected AST, using string dumps) is too hard to maintain.</p>
</div>
<div class="paragraph">
<p>Xpect test can be ignored by inserting a <code>!</code> between <code>XPECT</code> and the test name, e.g.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>// XPECT ! errors --&gt; '~$message$~' at "~$location$~"</code></pre>
</div>
</div>
<div class="sect3">
<h4 id="sec:Xpect_Test_Setup"><a class="anchor" href="#sec:Xpect_Test_Setup"></a><a class="link" href="#sec:Xpect_Test_Setup">14.6.1. Xpect Test Setup</a></h4>
<div class="paragraph">
<p>The setup is either defined in the file itself, e.g.,</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>/* XPECT_SETUP org.eclipse.n4js.spec.tests.N4JSSpecTest END_SETUP */</code></pre>
</div>
</div>
<div class="paragraph">
<p>or bundle-wide for a specific language in the plugin.xml (or fragment.xml), e.g.,</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>&lt;extension point="org.xpect.testSuite"&gt;
&lt;testSuite class="org.eclipse.n4js.spec.tests.N4JSSpecTest" fileExtension="n4js" /&gt;
&lt;/extension&gt;</code></pre>
</div>
</div>
</div>
<div class="sect3">
<h4 id="sec:Xpect_Issue_Suppression"><a class="anchor" href="#sec:Xpect_Issue_Suppression"></a><a class="link" href="#sec:Xpect_Issue_Suppression">14.6.2. Xpect Issue Suppression</a></h4>
<div class="paragraph">
<p>To configure an Xpect test class to suppress issues, you have to use the <code>@XpectImport</code> annotation to import the XpectSetupFactory <code>org.eclipse.n4js.xpect.validation.suppression.SuppressIssuesSetup</code>. Any Xpect test that is executed by this runner will work on the filtered list of issues.</p>
</div>
<div class="paragraph">
<p>Similar to issue suppressing JUnit tests, the suppressed issue codes are specified by<br>
<code>DEFAULT_SUPPRESSED_ISSUE_CODES_FOR_TESTS</code> constant in <code>N4JSLanguageConstants</code>.</p>
</div>
<div class="paragraph">
<p>For further per-file configuration a custom <code>XPECT_SETUP</code> parameter can be used. This overrides the suppression configuration of an Xpect runner class for the current file.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>/* XPECT_SETUP org.eclipse.n4js.tests.N4JSXpectTest
IssueConfiguration {
IssueCode "AST_LOCAL_VAR_UNUSED" {enabled=true}
}
END_SETUP
*/</code></pre>
</div>
</div>
<div class="paragraph">
<p>In this example the issue code <code>AST_LOCAL_VAR_UNUSED</code> is explicitly enabled which means that no issue with this issue code will be suppressed.</p>
</div>
</div>
<div class="sect3">
<h4 id="sec:Xpect_Provided_Test_Methods"><a class="anchor" href="#sec:Xpect_Provided_Test_Methods"></a><a class="link" href="#sec:Xpect_Provided_Test_Methods">14.6.3. Xpect Provided Test Methods</a></h4>
<div class="sect4">
<h5 id="errors"><a class="anchor" href="#errors"></a><a class="link" href="#errors">14.6.3.1. errors</a></h5>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Definition
</td>
<td class="hdlist2">
<p>Single line:</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>// XPECT errors --&gt; '~$message$~' at "~$location$~"</code></pre>
</div>
</div>
<div class="paragraph">
<p>Multi line:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>/* XPECT errors ---
'~$message_1$~' at "~$location_1$~"
~$\dots$~
'~$message_n$~' at "~$location_n$~"
--- */</code></pre>
</div>
</div>
</td>
</tr>
</table>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Description
</td>
<td class="hdlist2">
<p>Checks that one or more errors are issued at given location and compares the actual messages at a given location with the expected messages specified in the test.</p>
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>Also see <code>no errors</code> below.</p>
</div>
</div>
<div class="sect4">
<h5 id="warnings"><a class="anchor" href="#warnings"></a><a class="link" href="#warnings">14.6.3.2. warnings</a></h5>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Definition
</td>
<td class="hdlist2">
<p>Single line:<br></p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>// XPECT warnings --&gt; '~$Message$~' at "~$Location$~"</code></pre>
</div>
</div>
<div class="paragraph">
<p>Multi line:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>/* XPECT warnings ---
'~$message_1$~' at "~$location_1$~"
~$\dots$~
'~$message_n$~' at "~$location_n$~"
--- */</code></pre>
</div>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
Description
</td>
<td class="hdlist2">
<p>Checks that one or more warnings are issued at given location and compares the actual messages at a given location with the expected messages specified in the test.</p>
</td>
</tr>
</table>
</div>
</div>
</div>
<div class="sect3">
<h4 id="sec:N4JS_Specific_Xpect_Test_Methods"><a class="anchor" href="#sec:N4JS_Specific_Xpect_Test_Methods"></a><a class="link" href="#sec:N4JS_Specific_Xpect_Test_Methods">14.6.4. N4JS Specific Xpect Test Methods</a></h4>
<div class="paragraph">
<p>There are a lot of N4 specific Xpect tests methods available. To get all of these methods, search for references to annotation <code>org.xpect.runner.Xpect</code> in the N4 test plugins.</p>
</div>
<div class="sect4">
<h5 id="sec:XPECT_noerrors"><a class="anchor" href="#sec:XPECT_noerrors"></a><a class="link" href="#sec:XPECT_noerrors">14.6.4.1. noerrors and nowarnings</a></h5>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Definition
</td>
<td class="hdlist2">
<p>Single line:</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>// XPECT noerrors --&gt; '~$messageOrComment$~' at "~$location$~"</code></pre>
</div>
</div>
<div class="paragraph">
<p>Multi line:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>/* XPECT noerrors ---
'~$messageOrComment_1$~' at "~$location_1$~"
~$\dots$~
'~$messageOrComment_n$~' at "~$location_n$~"
--- */</code></pre>
</div>
</div>
</td>
</tr>
</table>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Provided by
</td>
<td class="hdlist2">
<p><code>NoerrorsValidationTest</code></p>
</td>
</tr>
<tr>
<td class="hdlist1">
Description
</td>
<td class="hdlist2">
<p>Checks that at the given location <em>no</em> error (or warning) is issued. This tests is roughly speaker the opposite of <code>errors</code>. The idea behind this test is to replace comments in the code, stating that an expression is assumed to be valid, with an explicit test. This is in particular useful when you start working on a task, in which there are (wrong) errors at a given position, or for bug report.</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Example
</td>
<td class="hdlist2">
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>function foo(any o): number {
if (o instanceof string) {
// XPECT noerrors --&gt; "effect systems knows that o is a string" at "o"
return o.length;
}
return 0;
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>is clearer and more explicit than</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>function foo(any o): number {
if (o instanceof string) {
// here should be no error:
return o.length;
}
return 0;
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>Also, the <code>noerrors</code> version will fail with a correct description, while the second one would fail with a general error and no location. Once the feature is implemented, regressions are detected much easier with the explicit version.</p>
</div>
</div>
</div>
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="sec:XPECT_scope"><a class="anchor" href="#sec:XPECT_scope"></a><a class="link" href="#sec:XPECT_scope">14.6.4.2. scope</a></h5>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Definition
</td>
<td class="hdlist2">
<p>Single line:</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>// XPECT scope at $location$ --&gt; ~$[$~!~$]$~~$name_1$~, ~$\dots$~, ~$[$~!~$]$~~$name_n$~ ~$[$ ~, ...~$]$~</code></pre>
</div>
</div>
<div class="paragraph">
<p>Multi line:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>/* XPECT scope $location$ ---
~$[$~!~$]$~~$name_1$~, ~$\dots$~,
~$[$~!~$]$~~$name_n$~~$[$ ~, ...~$]$~
--- */</code></pre>
</div>
</div>
</td>
</tr>
</table>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Provided by
</td>
<td class="hdlist2">
<p><code>PositionAwareScopingXpectTest</code></p>
</td>
</tr>
<tr>
<td class="hdlist1">
Description
</td>
<td class="hdlist2">
<p>Checks that the expected elements are actually found in the scope (or explicitly not found, when <code>!</code> is used). This is a modified version of the Xpect built-in scope test, ensuring that also elements only put into the scope when they are explicitly requested are found.</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Example
</td>
<td class="hdlist2">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>// XPECT scope at 'this.|$data_property_b' --&gt; a, b, $data_property_b, !newB, ...
return this.$data_property_b + "_getter";</code></pre>
</div>
</div>
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="sec:XPECT_scopeWithPosition"><a class="anchor" href="#sec:XPECT_scopeWithPosition"></a><a class="link" href="#sec:XPECT_scopeWithPosition">14.6.4.3. scopeWithPosition</a></h5>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Definition
</td>
<td class="hdlist2">
<p>Single line:</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-n4js" data-lang="n4js">// XPECT scopeWithPosition at $location$ --&gt; ~$[$~!~$]$~~$name_1$~ - ~$pos_1$~, ~$\dots$~, ~$[$~!~$]$~~$name_n$~ - ~$pos_n$~ ~$[$ ~, ...~$]$~</code></pre>
</div>
</div>
<div class="paragraph">
<p>Multi line:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-n4js" data-lang="n4js">/* XPECT scopeWithPosition $location$ ---
~$[$~!~$]$~~$name_1$~ - ~$pos_1$~, ~$\dots$~,
~$[$~!~$]$~~$name_n$~ - ~$pos_n$~ ~$[$ ~, ...~$]$~
--- */</code></pre>
</div>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
Provided by
</td>
<td class="hdlist2">
<p><code>PositionAwareScopingXpectTest</code></p>
</td>
</tr>
<tr>
<td class="hdlist1">
Description
</td>
<td class="hdlist2">
<p>Checks that the expected elements are actually found in the scope (or explicitly not found, when <code>!</code> is used). The concrete syntax of the position, which is usually the line number, or the line number prefix with <code>T</code> if a type element is referenced, is described in <code>EObjectDescriptionToNameWithPositionMapper</code>.</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Example
</td>
<td class="hdlist2">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>/* XPECT scopeWithPosition at foo2 ---
b - 9,
c - 25,
foo - T3,
foo2 - T9,
...
---*/
foo2()</code></pre>
</div>
</div>
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="sec:XPECT_scopeWithResource"><a class="anchor" href="#sec:XPECT_scopeWithResource"></a><a class="link" href="#sec:XPECT_scopeWithResource">14.6.4.4. scopeWithResource</a></h5>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Definition
</td>
<td class="hdlist2">
<p>Single line:</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>//</code></pre>
</div>
</div>
<div class="paragraph">
<p>Multi line:</p>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
Provided by
</td>
<td class="hdlist2">
<p><code>N4JSXpectTest</code></p>
</td>
</tr>
<tr>
<td class="hdlist1">
Description
</td>
<td class="hdlist2">
<p>Compares scope including resource name but not line number.</p>
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="sec:XPECT_binding"><a class="anchor" href="#sec:XPECT_binding"></a><a class="link" href="#sec:XPECT_binding">14.6.4.5. binding</a></h5>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Definition
</td>
<td class="hdlist2">
<p>Single line:</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>//</code></pre>
</div>
</div>
<div class="paragraph">
<p>Multi line:</p>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
Provided by
</td>
<td class="hdlist2">
<p><code>N4JSXpectTest</code></p>
</td>
</tr>
<tr>
<td class="hdlist1">
Description
</td>
<td class="hdlist2">
<p>Checks that a given element is bound to something identified by (simple) qualified name. The check is designed as simple as possible. That is, simply the next following expression is tested, and within that we expect a property access or a direct identifiable element. The compared name is the simple qualified name, that is container (type) followed by elements name, without URIs of modules etc.</p>
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="sec:XPECT_linkedPathname"><a class="anchor" href="#sec:XPECT_linkedPathname"></a><a class="link" href="#sec:XPECT_linkedPathname">14.6.4.6. linkedPathname</a></h5>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Definition
</td>
<td class="hdlist2">
<p>Single line:</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>// XPECT linkedPathname at '$location$' --&gt; ~$pathname$~</code></pre>
</div>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
Provided by
</td>
<td class="hdlist2">
<p><code>LinkingXpectMethod</code></p>
</td>
</tr>
<tr>
<td class="hdlist1">
Description
</td>
<td class="hdlist2">
<p>Checks that an identifier is linked to a given element identified by its path name. The path name is the qualified name in which the segments are separated by ’/’. This test does not use the qualified name provider, as the provider may return null for non-globally available elements. It rather computes the name again by using reflection, joining all <code>name</code> properties of the target and its containers.</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Example
</td>
<td class="hdlist2">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>// XPECT linkedPathname at 'foo()' --&gt; C/foo
new C().foo();</code></pre>
</div>
</div>
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="sec:XPECT_type_of"><a class="anchor" href="#sec:XPECT_type_of"></a><a class="link" href="#sec:XPECT_type_of">14.6.4.7. type of</a></h5>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Definition
</td>
<td class="hdlist2">
<p>Single line:</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>// XPECT type of '$location$' --&gt; ~$type$~</code></pre>
</div>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
Provided by
</td>
<td class="hdlist2">
<p><code>N4JSXpectTest</code></p>
</td>
</tr>
<tr>
<td class="hdlist1">
Description
</td>
<td class="hdlist2">
<p>Checks that the type inferred at location is similar to expected type.</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Example
</td>
<td class="hdlist2">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>// XPECT type of 'x' --&gt; string
var x = 'hello';
// XPECT type of 'foo()' --&gt; union{A,number}
var any y = foo();</code></pre>
</div>
</div>
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="sec:XPECT_expectedType"><a class="anchor" href="#sec:XPECT_expectedType"></a><a class="link" href="#sec:XPECT_expectedType">14.6.4.8. expectedType</a></h5>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Definition
</td>
<td class="hdlist2">
<p>-</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Single line
</td>
<td class="hdlist2">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>// XPECT expectedType at 'location' --&amp;gt; Type</code></pre>
</div>
</div>
<div class="literalblock">
<div class="content">
<pre>The location (at) is optional.</pre>
</div>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
Provided by
</td>
<td class="hdlist2">
<p><code>N4JSXpectTest</code></p>
</td>
</tr>
<tr>
<td class="hdlist1">
Description
</td>
<td class="hdlist2">
<p>Checks that an element/expression has a certain expected type (i.e. Xsemantics judgment expectedTypeIn).</p>
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="sec:XPECT_elementKeyword"><a class="anchor" href="#sec:XPECT_elementKeyword"></a><a class="link" href="#sec:XPECT_elementKeyword">14.6.4.9. elementKeyword</a></h5>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Definition
</td>
<td class="hdlist2">
<p>Single line:</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>// XPECT elementKeyword at 'myFunction' -&gt; function</code></pre>
</div>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
Example
</td>
<td class="hdlist2">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>interface I {
fld: int;
get g(): string;
set s(p:string);
}
//XPECT elementKeyword at 'string' --&gt; primitive
var v1: string;
//XPECT elementKeyword at 'I' --&gt; interface
var i: I;
//XPECT elementKeyword at 'fld' --&gt; field
i.fld;</code></pre>
</div>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
Provided by
</td>
<td class="hdlist2">
<p><code>ElementKeywordXpectMethod</code></p>
</td>
</tr>
<tr>
<td class="hdlist1">
Description
</td>
<td class="hdlist2">
<p>Checks that an element/expression has a certain element keyword. The expected element keyword is identical
to the element keyword shown when hovering the mouse over that element/expression in the N4JS IDE. This method is particuarly useful for testing merged elements of union/intersection.</p>
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="sec:XPECT_accessModifier"><a class="anchor" href="#sec:XPECT_accessModifier"></a><a class="link" href="#sec:XPECT_accessModifier">14.6.4.10. accessModifier</a></h5>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Definition
</td>
<td class="hdlist2">
<p>Single line:</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>// XPECT accessModifier at 'myFunction' -&gt; function</code></pre>
</div>
</div>
<div class="paragraph">
<p>or</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>// XPECT accessModifier -&gt; function</code></pre>
</div>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
Example
</td>
<td class="hdlist2">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>// XPECT accessModifier --&gt; publicInternal
export @Internal public abstract class MyClass2 {
// XPECT accessModifier --&gt; project
abstract m1(): string;
// XPECT accessModifier at 'm2' --&gt; project
m2(): string {
return "";
}
}</code></pre>
</div>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
Provided by
</td>
<td class="hdlist2">
<p><code>AccessModifierXpectMethod</code></p>
</td>
</tr>
<tr>
<td class="hdlist1">
Description
</td>
<td class="hdlist2">
<p>Checks that an element/expression has a certain accessibility.</p>
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="sec:XPECT_compileResult"><a class="anchor" href="#sec:XPECT_compileResult"></a><a class="link" href="#sec:XPECT_compileResult">14.6.4.11. compileResult</a></h5>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Definition
</td>
<td class="hdlist2">
<p>Single line:</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>//</code></pre>
</div>
</div>
<div class="paragraph">
<p>Multi line:</p>
</div>
</td>
</tr>
</table>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Provided by
</td>
<td class="hdlist2">
<p>-</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Description
</td>
<td class="hdlist2">
<p><em>This test should only be used during development of compiler and not used in the long run, because this kind of test is extremely difficult to maintain.</em></p>
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="sec:XPECT_output"><a class="anchor" href="#sec:XPECT_output"></a><a class="link" href="#sec:XPECT_output">14.6.4.12. output</a></h5>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Definition
</td>
<td class="hdlist2">
<p>Single line:</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>//</code></pre>
</div>
</div>
<div class="paragraph">
<p>Multi line:</p>
</div>
</td>
</tr>
</table>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Provided by
</td>
<td class="hdlist2">
<p>-</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Description
</td>
<td class="hdlist2">
<p>The most important test for compiler/transpiler, but also for ensuring that N4JS internal validations and assumptions are true at runtime.</p>
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="sec:XPECT_outputRegEx"><a class="anchor" href="#sec:XPECT_outputRegEx"></a><a class="link" href="#sec:XPECT_outputRegEx">14.6.4.13. outputRegEx</a></h5>
</div>
<div class="sect4">
<h5 id="sec:XPECT_calculatedAccessModifier"><a class="anchor" href="#sec:XPECT_calculatedAccessModifier"></a><a class="link" href="#sec:XPECT_calculatedAccessModifier">14.6.4.14. calculatedAccessModifier</a></h5>
</div>
<div class="sect4">
<h5 id="sec:XPECT_spec"><a class="anchor" href="#sec:XPECT_spec"></a><a class="link" href="#sec:XPECT_spec">14.6.4.15. spec</a></h5>
</div>
<div class="sect4">
<h5 id="sec:XPECT_deadCode"><a class="anchor" href="#sec:XPECT_deadCode"></a><a class="link" href="#sec:XPECT_deadCode">14.6.4.16. deadCode</a></h5>
</div>
<div class="sect4">
<h5 id="sec:XPECT_returnOrThrows"><a class="anchor" href="#sec:XPECT_returnOrThrows"></a><a class="link" href="#sec:XPECT_returnOrThrows">14.6.4.17. returnOrThrows</a></h5>
</div>
<div class="sect4">
<h5 id="sec:XPECT_lint"><a class="anchor" href="#sec:XPECT_lint"></a><a class="link" href="#sec:XPECT_lint">14.6.4.18. lint</a></h5>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Definition
</td>
<td class="hdlist2">
<p>Single line:</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>/* XPECT lint */</code></pre>
</div>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
Provided by
</td>
<td class="hdlist2">
<p><code>CompileAndLintTest</code></p>
</td>
</tr>
<tr>
<td class="hdlist1">
Description
</td>
<td class="hdlist2">
<p>Passes the generated code through the JSHint JavaScript linter. This test includes for instance checking for missing semicolons and undefined variables. The whole test exclusively refers to the generated javascript code.</p>
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="sec:XPECT_lintFails"><a class="anchor" href="#sec:XPECT_lintFails"></a><a class="link" href="#sec:XPECT_lintFails">14.6.4.19. lintFails</a></h5>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Definition
</td>
<td class="hdlist2">
<p>Single line:</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>/* XPECT lintFails */</code></pre>
</div>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
Provided By
</td>
<td class="hdlist2">
<p><code>CompileAndLintTest</code></p>
</td>
</tr>
<tr>
<td class="hdlist1">
Description
</td>
<td class="hdlist2">
<p>Negation of lint. Fails on linting success. Expects linting errors.</p>
</td>
</tr>
</table>
</div>
</div>
</div>
<div class="sect3">
<h4 id="sec:FIXME_Xpect_modifier"><a class="anchor" href="#sec:FIXME_Xpect_modifier"></a><a class="link" href="#sec:FIXME_Xpect_modifier">14.6.5. FIXME Xpect modifier</a></h4>
<div class="paragraph">
<p>A modification of the official Xpect framework allows us to use the FIXME modifier on each test. <sup class="footnote">[<a id="_footnoteref_15" class="footnote" href="#_footnote_15" title="View footnote.">15</a>]</sup></p>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Syntax
</td>
<td class="hdlist2">
<p>FIXME can be applied on any test just after the XPECT keyword:</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>// XPECT FIXME xpectmethod ... --&gt; ...</code></pre>
</div>
</div>
<div class="paragraph">
<p>Tests will still be ignored if an exclamation mark (!) is put between XPECT and FIXME.</p>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
Description
</td>
<td class="hdlist2">
<p>Using FIXME on a test negates the result of the underlying JUnit test framework. Thus a failure will be reported as a <code>true assertion</code> and an assertion that holds will be reported as <code>failure</code> . This enables to author valuable tests of behaviour, which is still not functional.</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Example
</td>
<td class="hdlist2">
<p>For instance, if we encounter an error-message at a certain code position, but the code is perfectly right, then we have an issue. We can annotate the situation with a ’fix me’ ’noerrors’ expectation:</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>// Perfectly right behaviour XPECT FIXME noerrors --&gt;
console.log("fine example code with wrong error marker here.");</code></pre>
</div>
</div>
<div class="paragraph">
<p>This turns the script into an Xpect test. We can integrate the test right away into our test framework and it will not break our build (even though the bug is not fixed).</p>
</div>
<div class="paragraph">
<p>When the issue will be worked on, the developer starts with removing ’FIXME’ turning this into useful unit-test.</p>
</div>
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>It is crucial to understand that FIXME negates the whole assertion. Example: If one expects an error marker at a certain position using the ’errors’ directive, one must give the exact wording of the expected error-message to actually get the FIXME behaviour working. To avoid strange behaviour it is useful to describe the expected error a comment in front of the expectation and leave the message-section empty.</p>
</div>
</div>
<div class="sect3">
<h4 id="sec:Expectmatrix_Xpect_Test_Methods"><a class="anchor" href="#sec:Expectmatrix_Xpect_Test_Methods"></a><a class="link" href="#sec:Expectmatrix_Xpect_Test_Methods">14.6.6. Expectmatrix Xpect tests</a></h4>
<div class="paragraph">
<p>Applying test-driven development begins with authoring acceptance and functional tests for the work in the current sprint. By this the overall code quality is ensured for the current tasks to solve. Rerunning all collected tests with each build ensures the quality of tasks solved in the past. Currently there is no real support for tasks, which are not in progress but are known to be processed in the near or far future. Capturing non-trivial bug reports and turning them into reproducable failing test-cases is one example.</p>
</div>
<div class="paragraph">
<p>Usually people deactivate those future-task-tests in the test code by hand. This approach doesn’t allow to calculate any metrics about the code. One such metric would be: Is there any reported bug solved just by working on an (seemingly unrelated) scheduled task?</p>
</div>
<div class="paragraph">
<p>To achieve measurements about known problems, a special build-scenario is set up. As a naming convention all classes with names matching <code>* Pending</code> are assumed to be Junit tests. In bundle <code>org.eclipse.n4js.expectmatrix.tests</code> two different Xpect-Runners are provided, each working on its own subfolder. Usual Xpect-tests are organised in folder xpect-test while in folder xpect-pending all future-tests are placed. A normal maven-build processes only the standard junit and xpect tests. Starting a build with profile <code>execute-expectmatrix-pending-tests</code> will additionally execute Xpect tests from folder xpect-pending and for all bundles inheriting from <code>/tests/pom.xml</code> all unit tests ending in <code>* Pending</code>. This profile is deactivated by default.</p>
</div>
<div class="paragraph">
<p>A special jenkins-job - N4JS-IDE_nightly_bugreports_pending - is configured to run the pending tests and render an overview und history to compare issues over time. Due to internal Jenkins structures this build always marked failed, even though the maven-build succeeds successfully.</p>
</div>
<div class="paragraph">
<p>Relevant additional information can be found in</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Jenkins job for pending cases: <a href="http://build-master:8080/view/N4JS/job/N4JS-IDE_nightly_bugreports_pending/" class="bare">http://build-master:8080/view/N4JS/job/N4JS-IDE_nightly_bugreports_pending/</a></p>
</li>
<li>
<p>Testmatrix <a href="https://docs.google.com/a/numberfour.eu/spreadsheets/d/1Blo58cRwIWemaiBNSnOtsQ8U7b3FoodX3yEs7oJKIg0/" class="bare">https://docs.google.com/a/numberfour.eu/spreadsheets/d/1Blo58cRwIWemaiBNSnOtsQ8U7b3FoodX3yEs7oJKIg0/</a></p>
</li>
</ul>
</div>
</div>
<div class="sect3">
<h4 id="xpect-lint-tests"><a class="anchor" href="#xpect-lint-tests"></a><a class="link" href="#xpect-lint-tests">14.6.7. Xpect Lint Tests</a></h4>
<div id="sec:XPECT_Lint_Tests" class="imageblock">
<div class="content">
<img src="chapters/15_tests/images/diag_XpectLint.svg" alt="diag XpectLint">
</div>
<div class="title">Figure 46. Xpect Lint</div>
</div>
<div class="paragraph">
<p>The test transpiles the provided n4js resource and checks the generated code. This is achieved using the Javascript linter JSHint.</p>
</div>
<div class="paragraph">
<p>After transpiling the provided n4js resource the LintXpectMethod combines the generated code with the jshint code into a script. It calls the JSHint validation function and returns the linting result as a json object. The error results are displayed in the console. The script is executed using the <code>Engine</code> class. (<a href="#sec:Design">Design</a>)</p>
</div>
<div class="paragraph">
<p>For the linting process an adapted configuration for JSHint is used. For the needs of N4JS the linter is configured to recognise N4JS specific globals. Details about the error codes can be found at the <a href="https://github.com/jshint/jshint/blob/2444a0463e1a99d46e4afa50ed934c317265529d/src/messages.js">jshint repository</a>.</p>
</div>
<div class="paragraph">
<p>The following warnings are explicitly enabled/disabled:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><strong>W069</strong>: [’a’] is better written in dot notation <strong>DISABLED</strong></p>
</li>
<li>
<p><strong>W033</strong>: Missing semicolon <strong>ENABLED</strong></p>
</li>
<li>
<p><strong>W014</strong>: Bad line breaking before ’a’. <strong>DISABLED</strong></p>
</li>
<li>
<p><strong>W032</strong>: Uneccesarry semicolon <strong>ENABLED</strong></p>
</li>
<li>
<p><strong>W080</strong>: It’s not necessary to initialize ’a’ to ’undefined’. <strong>ENABLED</strong></p>
</li>
<li>
<p><strong>W078</strong>: Setter is defined without getter. <strong>DISABLED</strong></p>
</li>
<li>
<p>ES6 related warnings are <strong>disabled</strong> using the ’esnext’ option: <strong>W117</strong>: Symbol is not defined. <strong>DISABLED</strong> <strong>W104</strong>: ’yield’ is only available in ES6 <strong>DISABLED</strong> <strong>W117</strong>: Promise is not defined <strong>DISABLED</strong> <strong>W119</strong>: function* is only available in ES6 <strong>DISABLED</strong></p>
</li>
</ul>
</div>
<div class="paragraph">
<p>The xpect lint test only applies if the provided resource passes the n4js compiler.</p>
</div>
<div class="paragraph">
<p>The xpect method lintFails can be used to create negative tests. All linting issues discovered during the development of the xpect plugin have there own negative test to keep track of their existence.</p>
</div>
<div class="paragraph">
<p>Additional information:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>JSHint: <a href="http://jshint.com/docs/" class="bare">http://jshint.com/docs/</a></p>
</li>
</ul>
</div>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="xpect-proposal-tests"><a class="anchor" href="#xpect-proposal-tests"></a><a class="link" href="#xpect-proposal-tests">14.7. Xpect Proposal Tests</a></h3>
<div class="paragraph">
<p>Proposal tests are all tests which verify the existence and application of completion proposals, created by content assist, quick fixes etc.</p>
</div>
<div class="paragraph">
<p>The key attributes of a proposal (cf <code>ConfigurableCompletionProposal</code>) are:</p>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
displayString
</td>
<td class="hdlist2">
<p>the string displayed in the proposal list</p>
</td>
</tr>
<tr>
<td class="hdlist1">
replacementString
</td>
<td class="hdlist2">
<p>simple variant of which is to be added to document, not necessarily the whole replacement (as this may affect several locations and even user interaction)</p>
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>In the tests, a <em>proposal is identified by a string contained in the displayString</em>. If several proposal match, test will fail (have to rewrite test setup or proposal identifier to be longer). Proposal identifier should be as short as possible (to make test robust), but not too short (to make test readable).</p>
</div>
<div class="paragraph">
<p>The following proposal tests are defined:</p>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
contentAssist <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mo>[</mo></mrow></math> List <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mo>]</mo></mrow></math>
</td>
<td class="hdlist2">
<p>verifies proposals created by content assist</p>
</td>
</tr>
<tr>
<td class="hdlist1">
quickFix <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mo>[</mo></mrow></math> List <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mo>]</mo></mrow></math>
</td>
<td class="hdlist2">
<p>verifies proposals created by quick fixes. Cursor position is relevant, that’s handled by the framework. We only create tests with cursor position – fixes applied via the problem view should work similarly, but without final cursor position.</p>
<div class="paragraph">
<p>If no error is found at given position, test will fail (with appropriate error message!). In call cases of apply, the issue must be resolved. Usually, fixing an issue may leave the file invalid as other issues still exists, or because by fixing one issue others may be introduced (which may happen often as we try to avoid consequential errors in validation). For some special cases, quickFix tests support special features, see below.</p>
</div>
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>Not tested in this context: Verify proposal description, as these tests would be rather hard to maintain and the descriptions are often computed asynchronously.</p>
</div>
<div class="sect3">
<h4 id="sec:Validation_vs__Non_Validation"><a class="anchor" href="#sec:Validation_vs__Non_Validation"></a><a class="link" href="#sec:Validation_vs__Non_Validation">14.7.1. Validation vs. Non-Validation</a></h4>
<div class="paragraph">
<p>We expect proposal tests to be applied on non-valid test files, and usually file is also broken after a proposal has been applied. Thus, the test-suite must not fail if the file is not valid.</p>
</div>
</div>
<div class="sect3">
<h4 id="sec:General_Proposal_Test_Features"><a class="anchor" href="#sec:General_Proposal_Test_Features"></a><a class="link" href="#sec:General_Proposal_Test_Features">14.7.2. General Proposal Test Features</a></h4>
<div class="sect4">
<h5 id="sec:Test_Variables"><a class="anchor" href="#sec:Test_Variables"></a><a class="link" href="#sec:Test_Variables">14.7.2.1. Test Variables</a></h5>
<div class="paragraph">
<p>Often, list of proposals are similar for different tests (which define different scenarios in which the proposals should be generated). For that reason, variables can be defined in the test set up:</p>
</div>
<div class="paragraph">
<p>In the Xpect-Setup there is now a special <code>Config</code> component where specific switches are accessible. For instance the timeout switch for content assist can be modified:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>/* XPECT_SETUP org.eclipse.n4js.tests.N4JSNotValidatingXpectPluginUITest
...
Config {
content_assist_timeout = 1000
...
}
...
*/</code></pre>
</div>
</div>
<div class="paragraph">
<p>Note: There should only be one <code>Config</code> component per Xpect-Setup.</p>
</div>
<div class="paragraph">
<p>Variables are introduced via the <code>VarDef</code> component. It takes a string argument as the variable name on construction. Inside the body one add <code>MemberLists</code> and <code>StringLists</code> arguments. Variable definitions may appear in <code>Config</code> bodies or in the Xpect-Setup.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>VarDef "objectProposals" {
...
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>Define variables with expression: A simple selector is given with the <code>MemberList</code> component. These components take three <code>String</code> arguments in the constructor. The first one is a typename. The second one is the feature selector, e.g. <code>methods</code> , <code>fields</code> , …and the third one defines the visibility.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>/* XPECT_SETUP
VarDef "stringProposals" { MemberList "String" "methods" "public" {}}
END_SETUP */</code></pre>
</div>
</div>
<div class="paragraph">
<p>We have to define a filter later in Xtend/Java, e.g., <code>getClassWithName( <em>className</em> ).filterType(<em>methods</em>).filterVisibility(<em>accessodifier</em>)&#8230;&#8203;</code></p>
</div>
<div class="paragraph">
<p>A variable is later referenced as follows:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>&lt;$variable&gt;</code></pre>
</div>
</div>
<div class="paragraph">
<p>Usage example:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>// XPECT contentAssistList at 'a.&lt;|&gt;methodA' proposals --&gt; &lt;$stringProposals&gt;, methodA2
a.methodA</code></pre>
</div>
</div>
</div>
<div class="sect4">
<h5 id="sec:Location_and_Selection"><a class="anchor" href="#sec:Location_and_Selection"></a><a class="link" href="#sec:Location_and_Selection">14.7.2.2. at – Location and Selection</a></h5>
<div class="paragraph">
<p>Tokens in expectation/setup:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>&lt;|&gt;</code> cursor position</p>
</li>
<li>
<p><code>&lt;[&gt;</code> selection start → also defines cursor position</p>
</li>
<li>
<p><code>&lt;]&gt;</code> selection end</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>All proposal tests have to specify a location via <code>at</code>, the location must contain the cursor position and may contain a selection. E.g.:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>// XPECT contentAssistList at 'a.&lt;|&gt;methodA' apply 'methodA2' --&gt; a.methodA2()&lt;|&gt;methodA
// XPECT contentAssistList at 'a.&lt;|&gt;&lt;[&gt;methodA&lt;]&gt;' apply 'methodA2' override --&gt; a.methodA2()&lt;|&gt;
a.methodA</code></pre>
</div>
</div>
</div>
<div class="sect4">
<h5 id="sec:Multi_Line_Expectations_in_Proposal_Tests"><a class="anchor" href="#sec:Multi_Line_Expectations_in_Proposal_Tests"></a><a class="link" href="#sec:Multi_Line_Expectations_in_Proposal_Tests">14.7.2.3. Multi Line Expectations in Proposal Tests</a></h5>
<div class="paragraph">
<p>In multiline expectations, ignored lines can be marked with <code>&lt;&#8230;&#8203;&gt;</code>. This means that 0 to n lines may occur but are ignored for comparison.</p>
</div>
<div class="paragraph">
<p>All multiline expectations are compared line-wise, with exact match except line delimiters (which are ignored as well)</p>
</div>
</div>
<div class="sect4">
<h5 id="sec:Timeout"><a class="anchor" href="#sec:Timeout"></a><a class="link" href="#sec:Timeout">14.7.2.4. Timeout and Performance</a></h5>
<div class="paragraph">
<p>We define a default timeout for content assist tests. In set up, this timeout may be changed:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-n4js" data-lang="n4js">/* XPECT SETUP
...
content_assist_timeout = 2000ms
...
END_SETUP */</code></pre>
</div>
</div>
<div class="paragraph">
<p>Performance is measured by measuring the runtime of tests, we should later setup performance measurements similar to the performance tests.</p>
</div>
</div>
</div>
<div class="sect3">
<h4 id="proposals-verify-existence-of-proposals"><a class="anchor" href="#proposals-verify-existence-of-proposals"></a><a class="link" href="#proposals-verify-existence-of-proposals">14.7.3. proposals – Verify Existence of Proposals</a></h4>
<div class="paragraph">
<p>In general, one could verify if certain proposals are present or not present, and in which order they are present. This is verified by the <code>proposals</code> argument.</p>
</div>
<div class="paragraph">
<p>E.g.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>// XPECT contentAssistList at 'a.&lt;|&gt;methodA' proposals --&gt; &lt;$stringProposals&gt;, methodA2
a.methodA</code></pre>
</div>
</div>
<div class="paragraph">
<p>Additional flags:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Order modifier:</p>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
unordered
</td>
<td class="hdlist2">
<p>by default</p>
</td>
</tr>
<tr>
<td class="hdlist1">
ordered
</td>
<td class="hdlist2">
<p>the given expectations have to have that order, between these expectations other proposals may be present (in case of contains)</p>
</td>
</tr>
</table>
</div>
</li>
<li>
<p>Subset modifier:</p>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
exactly
</td>
<td class="hdlist2">
<p>(default, maybe changed later) no other expectations as the given ones (usually <code>contains</code> is recommended).</p>
</td>
</tr>
<tr>
<td class="hdlist1">
contains
</td>
<td class="hdlist2">
<p>at least the given expectations must be present, but others may be there as well.</p>
<div class="paragraph">
<p>Using <em>contains</em> must be used with care since we match items by searching for a proposal which contains one of the expected strings as a substring. So if the only available proposal were ’methodA2’ and we would test if proposals contain ’methodA’, ’methodA2’ we would obtain a passing test.</p>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
not
</td>
<td class="hdlist2">
<p>any of the given proposals must be NOT be proposed</p>
</td>
</tr>
</table>
</div>
</li>
</ul>
</div>
</div>
<div class="sect3">
<h4 id="sec:Verify_displayed_string"><a class="anchor" href="#sec:Verify_displayed_string"></a><a class="link" href="#sec:Verify_displayed_string">14.7.4. display – Verify displayed string</a></h4>
<div class="paragraph">
<p>We do not verify the text style. We only verify text:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>// XPECT contentAssistList at 'a.&lt;|&gt;methodA' display 'methodA2' --&gt; 'methodA2(): any - A'
a.methodA</code></pre>
</div>
</div>
<div class="paragraph">
<p>This kind of test should be only applied for few scenarios, because the long display tests are rather hard to maintain.</p>
</div>
</div>
<div class="sect3">
<h4 id="sec:Apply_Proposal"><a class="anchor" href="#sec:Apply_Proposal"></a><a class="link" href="#sec:Apply_Proposal">14.7.5. apply – Apply Proposal</a></h4>
<div class="paragraph">
<p>Execution of proposal, the expectation describes the expected text result. The tests follow the naming convention of Ending in …Application.</p>
</div>
<div class="paragraph">
<p>Additional flags:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>insertion mode</p>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
insert
</td>
<td class="hdlist2">
<p>(default) Ctrl not pressed, proposal is inserted at given location</p>
</td>
</tr>
<tr>
<td class="hdlist1">
override
</td>
<td class="hdlist2">
<p>Ctrl is pressed, proposal overrides selection.</p>
</td>
</tr>
</table>
</div>
</li>
</ul>
</div>
<div class="paragraph">
<p>Single line assumes change at line of initial cursor position:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>// XPECT contentAssist at 'a.&lt;|&gt;methodA' apply 'methodA2' --&gt; a.methodA2()methodA
// XPECT contentAssist at 'a.&lt;|&gt;methodA' apply 'methodA2' insert --&gt; a.methodA2()methodA
// XPECT contentAssist at 'a.&lt;|&gt;methodA' apply 'methodA2' override --&gt; a.methodA2()
a.methodA</code></pre>
</div>
</div>
<div class="paragraph">
<p>or</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>// XPECT quickFix at 'a.&lt;|&gt;method' apply 'methodA2' --&gt; a.methodA2();
a.methodTypo();</code></pre>
</div>
</div>
<div class="paragraph">
<p>Multiline expectations describe changes to the whole. In order to match the expectation context information around the relevant places must be given. The format is similar to a unified diff with a special rule for inline-changes. The comparison works in a line-based mode. Each line in the expectation is prefixed with one character of ’+’, ’|’, ’-’ or ’ ’. Inserted lines are marked with + and removed lines with -. Lines marked with | denote changes in the line. Difference is placed inline inside of a pair of square brackets with a | separating the removal on the left and the addition on the right.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>/* XPECT contentAssistList at 'a.me&lt;|&gt;thodA' apply 'methodA2' ---
&lt;...&gt;
import A from "a/b/c"
&lt;...&gt;
a.methodA2()&lt;|&gt;methodA
&lt;...&gt;
--- */
a.methodA</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>/* XPECT contentAssistList at 'a.me&lt;|&gt;thodA' apply 'methodA2' ---
import B from "p/b"
+import A from "a/b/c"
...
foo() {
|a.[me|thodA2()]
}
...
--- */
a.methodA</code></pre>
</div>
</div>
<div class="sect4">
<h5 id="resource-application-in-other-files"><a class="anchor" href="#resource-application-in-other-files"></a><a class="link" href="#resource-application-in-other-files">14.7.5.1. resource – application in other files</a></h5>
<div class="paragraph">
<p>The resource parameter is available for the quickfix xpect method. It specifies the target resource of the quickfix. (e.g. change declaration of type in another file to quickfix an issue).</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>/* XPECT quickFix at 'sameVendor.protected&lt;|&gt;Method()' apply 'Declare member as public, @Internal' resource='../../otherProject/src/other_file.n4js' ---
diff here
---
*/</code></pre>
</div>
</div>
<div class="paragraph">
<p>The syntax is similar to a normal multiline quickfix xpect test besides the addition of the resource parameter. The relative path points to a file in the same workspace as the test file. Note that the path refers to the folder structure specified in the XPECT SETUP header. It is relative to the folder the test file is contained in.<br>
<br>
The diff is between the specified resource before and after the application of the quickfix<br>
<br>
Note that the fileValid (<a href="#fileValidVerify-validation-status">Verify Validation Status</a>) parameter is not applicable to an extern resource.</p>
</div>
</div>
</div>
<div class="sect3">
<h4 id="sec:Content_Assist_Cycling"><a class="anchor" href="#sec:Content_Assist_Cycling"></a><a class="link" href="#sec:Content_Assist_Cycling">14.7.6. kind – Content Assist Cycling</a></h4>
<div class="paragraph">
<p>We assume the default kind to be ’n4js’. It is possible to select a proposal kind in the test set up or via the argument <code>kind</code> in the test.</p>
</div>
<div class="paragraph">
<p>Select kind in test setup:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>/* XPECT_SETUP
content_assist_kind = 'recommenders'
END_SETUP */</code></pre>
</div>
</div>
<div class="paragraph">
<p>Select kind in test:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>// XPECT contentAssistList kind 'smart' at 'a.&lt;|&gt;methodA' display 'methodA2' --&gt; 'methodA2(): any - A'
a.methodA</code></pre>
</div>
</div>
</div>
<div class="sect3">
<h4 id="fileValidVerify-validation-status"><a class="anchor" href="#fileValidVerify-validation-status"></a><a class="link" href="#fileValidVerify-validation-status">14.7.7. fileValid – Verify validation status</a></h4>
<div class="paragraph">
<p>In some cases, in particular in case of quick fix tests, the file should be valid after the proposal has been applied. This is added by an additional argument <code>fileValid</code>.</p>
</div>
<div class="paragraph">
<p>E.g.,</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>// XPECT quickFix at 'a.&lt;|&gt;method' apply 'methodA2' fileValid --&gt; a.&lt;|&gt;methodA2();
a.methodTypo();</code></pre>
</div>
</div>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Apply_Proposal_And_Execute_Tests"><a class="anchor" href="#sec:Apply_Proposal_And_Execute_Tests"></a><a class="link" href="#sec:Apply_Proposal_And_Execute_Tests">14.8. Apply Proposal And Execute Tests</a></h3>
<div class="paragraph">
<p>If a proposal fixes all issues in a test file, the file could be executed. This is an important type of test, as this is what the user expects in the end. Besides, this type of test is very robust, as it does not depend on the concrete way how an issue is fixed. For quick fixes, these kind of tests are to be implemented!</p>
</div>
<div class="paragraph">
<p>The following combined proposal and execute tests are provided:</p>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
quickFixAndRun
</td>
<td class="hdlist2">
<p>applies a quick fix, verifies that all issues are solved, and executes the file.</p>
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>These tests are basically execution tests, that is the expectation describes the expected output of the script.</p>
</div>
<div class="paragraph">
<p>E.g.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>// XPECT quickFixAndRun at 'a.&lt;|&gt;method' apply 'methodHelloWorld' --&gt; Hello World
a.methodTypo();</code></pre>
</div>
</div>
<div class="paragraph">
<p>with <code>methodHelloWorld</code> printing <code>’Hello World’</code> to the console. The expected output can be multiline:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>/* XPECT quickFixAndRun at 'a.&lt;|&gt;method' apply 'methodHelloWorld' ---
Hello World
--- */
a.methodTypo();</code></pre>
</div>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Organize_Imports_Test"><a class="anchor" href="#sec:Organize_Imports_Test"></a><a class="link" href="#sec:Organize_Imports_Test">14.9. Organize Imports Test</a></h3>
<div class="paragraph">
<p>For testing organise imports a Plugin-UI test method is available. It operates in two modes. Either a successful application of organise imports is tested or the expected abortion is checked.</p>
</div>
<div class="sect3">
<h4 id="organizeimports"><a class="anchor" href="#organizeimports"></a><a class="link" href="#organizeimports">14.9.1. organizeImports</a></h4>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Definition
</td>
<td class="hdlist2">
<p>Multi line:</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>/* XPECT organizeImports ---
~Failure given by line-syntax starting with two characters:~
+ additional line
| line with inplace[removed part|added part]
+ removed line
unchanged line
--- */
// XPECT warnings --&gt; "The import of A is unused." at "A"
import A from "a/a1/A"
...
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>Single line:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>// XPECT organizeImports ambiguous '~Failure String of Exception~'--&gt;
}</code></pre>
</div>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
Provided by
</td>
<td class="hdlist2">
<p><code>OrganizeImportXpectMethod</code></p>
</td>
</tr>
<tr>
<td class="hdlist1">
Description
</td>
<td class="hdlist2">
<p>Checks that the resulting source-file differs in the described way. The Multiline variant checks the result of a successful application of organise import to the file. All errors and warnings prior to organising imports must be marked as the appropriate XPECT-error or warning.</p>
<div class="paragraph">
<p>The single-line notation checks the failure of an fully automatic reorganisation of the imports due to some reason. The reason is verified to be the given string after the <code>ambiguous</code> keyword. The expectation side (right of <code>-&#8594;</code>) is empty.</p>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
Example
</td>
<td class="hdlist2">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>/* XPECT_SETUP org.eclipse.n4js.tests.N4JSXpectPluginUITest
Workspace {
Project "P1" {
Folder "src" {
Folder "a" { Folder "a1" { File "A.n4js" { from="../../a/a1/A.n4js" } }
Folder "c" { ThisFile {} } } }
File "package.json" { from="package_p1.json" } } }
END_SETUP
*/
/* XPECT organizeImports ---
| import [A from "a/a1/A"|]
| [import|] AR from "a/a1/A"
export public role BRole with AR {
}
--- */
// XPECT warnings --&gt; "The import of A is unused." at "A"
import A from "a/a1/A"
import AR from "a/a1/A"
// XPECT noerrors --&gt; "Couldn't resolve reference to Type 'AR'."
export public role BRole with AR {
}</code></pre>
</div>
</div>
</td>
</tr>
</table>
</div>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Access_Control_Test"><a class="anchor" href="#sec:Access_Control_Test"></a><a class="link" href="#sec:Access_Control_Test">14.10. Access Control Test</a></h3>
<div class="paragraph">
<p>Access control refers to the decision whether or not a member or a top level element is accessible for a client. In this context, access refers to reading, writing, and calling a member or a top level function, and to overriding inherited members in classes and interfaces. In N4JS, access is controlled via modifiers at the type and at the member level. Due to the large number of such modifiers and the large number of different scenarios for access control, manually written tests can only cover a small number of actual scenarios. An automatic test generator helps to increase the test coverage for access control.</p>
</div>
<div class="paragraph">
<p>The test generator loads test scenarios from a CSV table that also contains the expected results of each test case and then generates N4JS projects and code for each test case, compiles them using the headless compiler, and compares the compilation results to the expectations from the CSV table. Note that the test generator does not generate test cases that check access control for top level functions and variables.</p>
</div>
<div class="sect3">
<h4 id="test-scenarios"><a class="anchor" href="#test-scenarios"></a><a class="link" href="#test-scenarios">14.10.1. Test Scenarios</a></h4>
<div class="paragraph">
<p>Each test scenario consists of a scenario specifier (one of <code>Extends</code> , <code>Implements</code> , <code>References</code> ), a supplier and a client (each of which can be a class, and abstract class, an interface, or an interface with default implementations of its getters, setters, and methods). Thereby, the client attempts to access a member of the supplier either by reading, writing, or calling it or by overriding it in the case of an <code>Extends</code> or <code>Implements</code> scenario. Furthermore, each test cases specifies the location of the client in relation to the location of the supplier, e.g., whether the client is in the same module, or whether it is in the same project (but not the same module), and so forth. Finally, a test case must specify the access control modifiers of the supplier type and the member that is being accessed by the client, whether that member is static or not, and, lastly, the type of access that is being attempted.</p>
</div>
</div>
<div class="sect3">
<h4 id="n4js-code-generator"><a class="anchor" href="#n4js-code-generator"></a><a class="link" href="#n4js-code-generator">14.10.2. N4JS Code Generator</a></h4>
<div class="paragraph">
<p>The test generator needs to generate N4JS projects and modules that implement a particular test case. For this purpose, it uses a simple code generator, available in the package <code>org.eclipse.n4js.tests.codegen</code>. The classes in that package allow specifying N4JS projects, modules, classes, and members in Java code and to generate the required artifacts as files at a given location.</p>
</div>
</div>
<div class="sect3">
<h4 id="xtext-issue-matcher"><a class="anchor" href="#xtext-issue-matcher"></a><a class="link" href="#xtext-issue-matcher">14.10.3. Xtext Issue Matcher</a></h4>
<div class="paragraph">
<p>To match the test expectations from the CSV file against the issues generated by the compiler, the test generator uses a small library of issue matchers, available in the package <code>org.eclipse.n4js.tests.issues</code>. The classes in that package make it easy to specify expectations against Xtext issues programmatically and to evaluate these expectations against a specific set of Xtext issues. Mismatches, that is, unexpected issues as well as unmatched expectations, are explained textually. These explanations are then shown in the failure notifications of the test case generator.</p>
</div>
<div class="paragraph">
<p>The test expectations allow for FIXME annotations in the CSV table to express cases where an access control error has been found, but hasn’t been fixed yet. Such expectations lead to inverted expectations against the generated issues: For example, if a test expects an attempted access to fail with an error, but the validation rules allow the access regardless, then a FIXME annotation at the test will invert the actual expectation: Where previously an error was expected, there must now be no error at all. Then, once the validation rules have been adjusted and the error occurs as expected, the FIXME test case will fail until the FIXME annotation has been removed. Therefore, the issue expectation matchers can be inverted and adjust their behavior accordingly.</p>
</div>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Smoke_Tests"><a class="anchor" href="#sec:Smoke_Tests"></a><a class="link" href="#sec:Smoke_Tests">14.11. Smoke Tests</a></h3>
<div class="paragraph">
<p>Smoke tests are useful to ensure the robustness of the IDE. They aim at revealing potential exceptions that may surface to the end user in the IDE or in a headless compile run. Therefore, different permutations of a given input document are fed into processing components such as the validator or the type system. No exceptions may be thrown from these components.</p>
</div>
<div class="paragraph">
<p>Since smoke tests are longrunning, it is not desired to execute them with each Maven run. That’s why the naming convention <code>* Smoketest</code> was choosen. It will not be matched by the naming pattern for normal JUnit tests during a maven run.</p>
</div>
<div class="paragraph">
<p>The smoke tests are generally created by using the (valid or invalid) input of existing test cases. Therefore, the a command line argument <code>-DSmokeTestWriter=true</code> may be passed to the VM that executes a test. All input documents that are processed by a <code>ParseHelper&lt;Script&gt;</code> will be written to a new test method on the console. The result can be merged manually into the <code>GeneraredSmoketest</code>.</p>
</div>
<div class="sect3">
<h4 id="how-to-handle-smoke-test-errors"><a class="anchor" href="#how-to-handle-smoke-test-errors"></a><a class="link" href="#how-to-handle-smoke-test-errors">14.11.1. How to handle smoke test errors?</a></h4>
<div class="paragraph">
<p>If a smoke test fails, the concrete input should be extracted into a dedicated error test case. The existing classes like <code>scoping.ErrorTest</code> should be used to add the broken input document and create fast running cases for them. For that purpose, the <code>ExceptionAnalyzer</code> can be used. Such a test case will usually look like this:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>@Test
def void testNoNPE_03() {
val script = `` '
var target = {
s: "hello",
set x
`` '.parse
analyser.analyse(script, "script", "script")
}</code></pre>
</div>
</div>
</div>
<div class="sect3">
<h4 id="smoketester-and-exceptionanalyzer"><a class="anchor" href="#smoketester-and-exceptionanalyzer"></a><a class="link" href="#smoketester-and-exceptionanalyzer">14.11.2. SmokeTester and ExceptionAnalyzer</a></h4>
<div class="paragraph">
<p>The components that are used to implemement the smoke tests are the <code>SmokeTester</code> and the <code>ExceptionAnalyzer</code>. The smoke tester performs the permutation of the characters from the input model whereas the analyzer does the heavy lifting of passing the parsed model to various components such as the type system or the validation. Both utilities can be used to add either new smoke test documents or to check for the robustness of an implementation. It’s espeically useful to use the ExceptionAnalyzer in conjunction with existing test suites like the <code>LibraryParsingTestSuite</code> since it can be used instead of the other utilities like the <code>PositiveAnalyzer</code> or the <code>NegativeAnalyzer</code>.</p>
</div>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:UI_Tests_with_SWTBot"><a class="anchor" href="#sec:UI_Tests_with_SWTBot"></a><a class="link" href="#sec:UI_Tests_with_SWTBot">14.12. UI Tests with SWTBot</a></h3>
<div class="paragraph">
<p>For testing functionality from the end-user perspective we use UI tests based on SWTBot <a href="http://eclipse.org/swtbot/" class="bare">http://eclipse.org/swtbot/</a>.</p>
</div>
<div class="paragraph">
<p>Since UI tests are usually rather fragile, it was decided to keep the number of these tests as low as possible. The main purpose of these tests is to ensure that the most fundamental IDE functionality is working properly, e.g. creating an N4JS project, creating a new N4JS file, running N4JS code in node.js.</p>
</div>
<div class="paragraph">
<p>The tests have a number of SWTBot dependencies. For details, please refer to the latest target platform definition file.</p>
</div>
<div class="sect3">
<h4 id="writing-swtbot-tests"><a class="anchor" href="#writing-swtbot-tests"></a><a class="link" href="#writing-swtbot-tests">14.12.1. Writing SWTBot Tests</a></h4>
<div class="paragraph">
<p>The implementation is contained in bundle <code>org.eclipse.swtbot.tests</code>. Writing SWTBot tests is straightforward; see source code of <code>AbstractSwtBotTest</code> for usage and examples.</p>
</div>
<div class="paragraph">
<p>Some hints:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Many methods of the SWTBot framework already include assertions. For example, the method <code># menu(String)</code> to find a particular menu or menu item will assert that the item was found and otherwise throw an exception. Therefore, it is safe to write code like:</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>bot.menu("File").menu("New").menu("Project...").click();</code></pre>
</div>
</div>
</li>
<li>
<p>It is usually considered bad practice to use sleep delays in UI tests. Instead, wait for the element to appear using methods such as <a href="http://download.eclipse.org/technology/swtbot/helios/dev-build/apidocs/org/eclipse/swtbot/swt/finder/SWTBot.html#waitUntil" class="bare">http://download.eclipse.org/technology/swtbot/helios/dev-build/apidocs/org/eclipse/swtbot/swt/finder/SWTBot.html#waitUntil</a> or <a href="http://download.eclipse.org/technology/swtbot/helios/dev-build/apidocs/org/eclipse/swtbot/swt/finder/SWTBot.html#waitWhile" class="bare">http://download.eclipse.org/technology/swtbot/helios/dev-build/apidocs/org/eclipse/swtbot/swt/finder/SWTBot.html#waitWhile</a>.</p>
</li>
</ul>
</div>
</div>
<div class="sect3">
<h4 id="running-swtbot-tests"><a class="anchor" href="#running-swtbot-tests"></a><a class="link" href="#running-swtbot-tests">14.12.2. Running SWTBot Tests</a></h4>
<div class="paragraph">
<p>To run the tests locally in Eclipse just right-click the bundle and select <code>Run As &gt; SWTBot Test</code> . To run them locally via maven simply start a normal maven build, no additional command line arguments, etc. required.</p>
</div>
<div class="paragraph">
<p>To run remotely in a Jenkins build: on Windows the build must be executed with a logged-in user; on Linux Xvfb and a window manager are required. The recommended window manager is metacity. More information can be found here: <a href="http://wiki.eclipse.org/SWTBot/Automate_test_execution" class="bare">http://wiki.eclipse.org/SWTBot/Automate_test_execution</a>.</p>
</div>
<div class="paragraph">
<p>To use metacity, add the following pre-build shell command to the Jenkins build configuration:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-bash" data-lang="bash">sleep 5; metacity --replace --sm-disable &amp;</code></pre>
</div>
</div>
<div class="paragraph">
<p>The sleep is required because metacity depends on Xvfb to be fully initialized, which might take a moment on slower build nodes. The following Jenkins console log shows the expected output when starting Xvfb and metacitiy:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-bash" data-lang="bash">Xvfb starting$ Xvfb :1 -screen 0 1024x768x24 -fbdir /var/lib/build/2014-09-05_10-36-343337290231975947044xvfb
[N4JS-IDE_Oliver] $ /bin/sh -xe /tmp/hudson4475531813520593700.sh
+ sleep 5
+ metacity --replace --sm-disable
Xlib: extension "RANDR" missing on display ":1.0".
Window manager warning: 0 stored in GConf key /desktop/gnome/peripherals/mouse/cursor_size is out of range 1 to 128
Process leaked file descriptors. See http://wiki.jenkins-ci.org/display/JENKINS/Spawning+processes+from+build for more information
Parsing POMs
...</code></pre>
</div>
</div>
<div class="paragraph">
<p>The warnings and error messages in the above log are expected and are considered unharmful (cf. discussion with Jörg Baldzer).</p>
</div>
</div>
</div>
<div class="sect2">
<h3 id="sec:Debugging_UI_Tests"><a class="anchor" href="#sec:Debugging_UI_Tests"></a><a class="link" href="#sec:Debugging_UI_Tests">14.13. Debugging UI Tests</a></h3>
<div class="paragraph">
<p>In rare cases UI Tests behave differently depending on the underlying OS und the power of the test machine. Missing user interaction on the build-server often results in waiting processes which in turn get a timeout request from the driving unit-testing-framework. To investigate the UI state on the build node a X11 connection needs to established.</p>
</div>
<div class="sect3">
<h4 id="sec:Connecting_to_the_X_server_on_the_build_node"><a class="anchor" href="#sec:Connecting_to_the_X_server_on_the_build_node"></a><a class="link" href="#sec:Connecting_to_the_X_server_on_the_build_node">14.13.1. Connecting to the X-server on the build-node</a></h4>
<div class="paragraph">
<p>First a vnc server needs to be started on the build-node. This is done via <code>x11vnc -display :1 -shared &gt;  /x11vnc.log 2&gt;&amp;1 &amp;</code> as a pre-build shellscript-step in Jenkins.</p>
</div>
<div class="paragraph">
<p>You can narrow down the build to run on a specific node in Jenkins if a repeatable environment is required otherwise the current build-node is listed in the overview page. For security reasons the connection needs to be tunneld through an ssh login:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-bash" data-lang="bash">ssh -l viewer -L 5900:localhost:5900 build-linux-25 ’x11vnc -localhost -display :1</code></pre>
</div>
</div>
<div class="paragraph">
<p>Here we are using the mating <code>build-linux-25</code> with the generic user/password ’viewer’ to start the program <code>x11vnc</code>. For the actual display number – <code>:1</code> in this case – you can refer to the console output. The command above tunnels the default vnc port 5900. You can now connect on <code>localhost</code> with a vnc client. If the user is not available, the <code>x11vnc</code> program not installed or in case of other issues, ask the build-and-release team.</p>
</div>
<div class="paragraph">
<p>To display the screen you can use any arbitrary vnc-client (on mac there is screen sharing, in theory just opened from the command line by hitting <code>open vnc://viewer:viewer@localhost:5900</code>). One working client is ’chicken of the vnc’ <a href="http://sourceforge.net/projects/cotvnc/" class="bare">http://sourceforge.net/projects/cotvnc/</a></p>
</div>
</div>
<div class="sect3">
<h4 id="sec:Tools_to_investigate_the_java_stack"><a class="anchor" href="#sec:Tools_to_investigate_the_java_stack"></a><a class="link" href="#sec:Tools_to_investigate_the_java_stack">14.13.2. Tools to investigate the java-stack</a></h4>
<div class="paragraph">
<p>Unix and Java come with some useful programs for investigations. The following tools may need some advanced rights to see processes from other users.</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>htop</code> enhanced top to see all currently running processes</p>
</li>
<li>
<p><code>jps</code> list running java processes</p>
</li>
<li>
<p><code>jstack</code> investigate running java process</p>
</li>
</ul>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_ui-concepts"><a class="anchor" href="#_ui-concepts"></a><a class="link" href="#_ui-concepts">15. UI Concepts</a></h2>
<div class="sectionbody">
<div class="admonitionblock warning">
<table>
<tr>
<td class="icon">
<i class="fa icon-warning" title=""></i>
</td>
<td class="content">
Parts of this chapter may be outdated.
</td>
</tr>
</table>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:User_Interface_Concepts"><a class="anchor" href="#sec:User_Interface_Concepts"></a><a class="link" href="#sec:User_Interface_Concepts">15.1. User Interface Concepts</a></h3>
<div class="sect3">
<h4 id="sec:Eclipse_UI_Concepts"><a class="anchor" href="#sec:Eclipse_UI_Concepts"></a><a class="link" href="#sec:Eclipse_UI_Concepts">15.1.1. Eclipse UI Concepts</a></h4>
<div class="paragraph">
<p>The following list gives an overview of Eclipse specific UI concepts and which classes are used for implementation.</p>
</div>
<div class="sect4">
<h5 id="sec:Label_Provider"><a class="anchor" href="#sec:Label_Provider"></a><a class="link" href="#sec:Label_Provider">15.1.1.1. Label Provider</a></h5>
<div class="paragraph">
<p>Also provides decorations for icons and text labels.</p>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Example
</td>
<td class="hdlist2">
<p>The representation of objects in the outline view or in search results.</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Eclipse API
</td>
<td class="hdlist2">
<div class="ulist">
<ul>
<li>
<p><code>o.e.jface.viewers.ILabelProvider</code> → without styes</p>
</li>
<li>
<p><code>o.e.jface.viewers.DelegatingStyledCellLabelProvider.IStyledLabelProvider</code> → with styles Drawback: Depends on Image rather than ImageDescriptor (see below)</p>
</li>
</ul>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
Xtext Specifics
</td>
<td class="hdlist2">
<div class="ulist">
<ul>
<li>
<p><code>Declarative API</code> via reflective, polymorphic dispatching. <code>org.eclipse.xtext.ui.label.AbstractLabelProvider</code></p>
</li>
<li>
<p>Allows to work with ImageDescriptors (non-ui-thread, can be composed), but cumbersome</p>
</li>
<li>
<p>DefaultLabelProvider will be used everywhere (outline etc.), returns the <code>name</code> (via reflection). You could bind specific label providers.</p>
</li>
<li>
<p>DescriptionLabelProvider provides labels for objects from the Xtext index used by <code>open model element</code> , <code>find references</code> – the later is already customized)</p>
</li>
</ul>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
Best Practices
</td>
<td class="hdlist2">
<p>Labels are often use-case specific, so a single label provider is not always useful Therefore, in Xtext are different label providers for different use cases. Use cases are defined and enumerated (see <code>DefaultUiModule.configure*Label*</code>).
+
Often labels could be easier created where they are needed instead of using a label provider (even for things like OutlineView). (LabelProvider are maybe over-engineered). (Note: default label provider has dependencies to SWT, because it uses images, which are often not needed; also they are only called with an object and no further configuration).
+
Images:
+</p>
<div class="ulist">
<ul>
<li>
<p>DeclarativeLabelProvider: text and image (String: path relative to icons folder; or: ImageDescriptor, or Image), put it into folder (needs to be called <code>icons</code> , otherwise bind another name)</p>
</li>
<li>
<p>better use AbstractLabelProvider and use EMF concepts (easier to debug and handle)</p>
</li>
<li>
<p>image format and size: sub-folders with 32, 12, 16, 24, etc. (look which sizes are needed); in about view maybe bigger; png (supports transparency)</p>
</li>
</ul>
</div>
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="sec:Markers"><a class="anchor" href="#sec:Markers"></a><a class="link" href="#sec:Markers">15.1.1.2. Markers</a></h5>
<div class="quoteblock">
<blockquote>
<div class="paragraph">
<p>Markers are objects that may be associated with Workbench resources. There are many uses of markers in the Workbench&#8230;&#8203;
Markers are shown in a marker view (Tasks, Problems or Bookmark view) or on the marker bar in the editor area.</p>
</div>
</blockquote>
<div class="attribution">
&#8212; Eclispe Help
</div>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Examples
</td>
<td class="hdlist2">
<p>Tasks, Problems, Bookmarks, Breakpoints, Trace information</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Eclipse API
</td>
<td class="hdlist2">
<p><code>org.eclipse.core.resources.IMarker</code> and <code>IResource.findMarkers</code>
+</p>
<div class="ulist">
<ul>
<li>
<p>Marker types registered via extension point (basically String). A marker is more or less a Map of String &#8594; String with some meta information, e.g. the resource location in WS, line numbers, type</p>
</li>
<li>
<p>Markers are very efficient (e.g., find markers of a certain type), cf Xtext Specifics</p>
</li>
<li>
<p>For validation, some new marker types are already registered.</p>
</li>
</ul>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
Xtext
</td>
<td class="hdlist2">
<p>(2.6)
+</p>
<div class="ulist">
<ul>
<li>
<p>Todo-Markers are created during build, task list is populated by these markers.</p>
</li>
<li>
<p><code>org.eclipse.xtext.tasks.ITaskFinder</code> (and default implementation is bound by default, can be replaced with custom implementation)</p>
</li>
</ul>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
Best Practices
</td>
<td class="hdlist2">
<p>In Xtend, markers are used to trace from original file to generated file. They are hidden (and not displayed), so in general markers can be used for non-UI-problems as well (but only available in Eclipse of course) clean up markers: no general solution, often managed by some (single) life-cycle aware class (e.g., Builder)</p>
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="sec:Commands__Toolbar_and_Menus"><a class="anchor" href="#sec:Commands__Toolbar_and_Menus"></a><a class="link" href="#sec:Commands__Toolbar_and_Menus">15.1.1.3. Commands, Toolbar and Menus</a></h5>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Examples
</td>
<td class="hdlist2">
<p>Organize Imports,</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Eclipse API
</td>
<td class="hdlist2">
<p>use Commands + Handlers instead of Actions (or ActionDelegates etc.)</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Xtext Specifics
</td>
<td class="hdlist2">
<p>uses commands and handlers</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Best Practice
</td>
<td class="hdlist2">
<p><strong>Use commands and handlers</strong> -
handler usually only delegates to real thing (that is, retrieve parameters from context and call the real thing)<br>
<strong>Register in pluginxml</strong> via ExecutableExtensionFactory to be able to use injection (also pre-generatd, e.g.:</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-xml" data-lang="xml"> &lt;handler class="org.eclipse.n4js.ui.N4JSExecutableExtensionFactory:org.eclipse.xtext.ui.editor.handler.ValidateActionHandler"
commandId="org.eclipse.n4js.N4JS.validate"&gt;</code></pre>
</div>
</div>
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p><strong>Undo</strong>: use TextEdit and MultiTextEdit (composed)<br>
otherwise very low level</p>
</div>
</div>
<div class="sect4">
<h5 id="sec:Content_Assist"><a class="anchor" href="#sec:Content_Assist"></a><a class="link" href="#sec:Content_Assist">15.1.1.4. Content Assist</a></h5>
<div class="quoteblock">
<blockquote>
<div class="paragraph">
<p>Content assist allows you to provide context sensitive content completion upon user request. Popup windows (infopops) are used to propose possible text choices to complete a phrase. The user can select these choices for insertion in the text. Content assist also supports contextual infopops for providing the user with information that is related to the current position in the document.</p>
</div>
</blockquote>
<div class="attribution">
&#8212; Eclipse Help
</div>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Risk
</td>
<td class="hdlist2">
<p>Always needs longer than anticipated.</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Examples
</td>
<td class="hdlist2">
<p>complete name of function in function call, complete keywords</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Eclipse API
</td>
<td class="hdlist2">
<div class="ulist">
<ul>
<li>
<p>IContentAssistant wraps the widget,</p>
</li>
<li>
<p>IContentProposalProvider computes the (array of) CompletionProposal (quite cumbersome!).</p>
</li>
<li>
<p>Many extension interfaces that provide valuable UI features.</p>
</li>
</ul>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
Xtext Specifics
</td>
<td class="hdlist2">
<div class="ulist">
<ul>
<li>
<p>ConfigurableCompletionProposal implements the currently defined extension interfaces, provides getters to modify the proposal after the fact.</p>
</li>
<li>
<p>Context: The ContentAssistContext is provided by the framework according to the current cursor position in the document (cf. <code>ContentAssisParser</code>), semantic context (semantic element) computed with best match strategy (worst case you get the root element). Multiple contexts may be valid at the very same cursor position since the replace region may be different for different proposals.</p>
</li>
<li>
<p>Various abstracts above the JFace stuff are available in Xtext, some of the <code>over the top</code> , others quite handy.</p>
</li>
</ul>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
Best Practices
</td>
<td class="hdlist2">
<div class="ulist">
<ul>
<li>
<p>List of follow elements can be supposed to be complete, no need to figure out them with regular expressions etc.</p>
</li>
<li>
<p>in rare cases it is necessary to <code>manually</code> scan the text context, e.g. to get the variable name based on the variable type. → we will provider a utility class for that using regex. NEVER search on the text with simple string methods.</p>
</li>
<li>
<p>In N4JSProposalProvider, override pre-generated methods (see <code>AbstractN4JSProposalProvider</code>) – do not overload (with concrete semantic element)</p>
</li>
<li>
<p>how to implement complete-methods:</p>
<div class="ulist">
<ul>
<li>
<p>inspect context, examine current semantic element provide elements from scope or hard coded proposal: see <a href="#sec:Proposals">Proposals</a></p>
</li>
</ul>
</div>
</li>
</ul>
</div>
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="sec:Quick_Fixes"><a class="anchor" href="#sec:Quick_Fixes"></a><a class="link" href="#sec:Quick_Fixes">15.1.1.5. Quick Fixes</a></h5>
<div class="quoteblock">
<blockquote>
Users can select a problem marker and choose a Quick Fix from a popup containing the list of supplied fixes contributed for the marker.
</blockquote>
<div class="attribution">
&#8212; Eclipse Help
</div>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Examples
</td>
<td class="hdlist2">
<p>Add Import, Add Override Annotation</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Eclipse API
</td>
<td class="hdlist2">
<p>Based on ICompletionProposal (powerful)
+</p>
<div class="ulist">
<ul>
<li>
<p>QuickFixes are registered to marker (marker attribute: is fixable or not – this attribute is a guess only, there does not need to be a quick fix)</p>
</li>
<li>
<p>MarkerResolutionGenerator (can also be used to fix several markers at once)</p>
</li>
</ul>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
Xtext Specifics
</td>
<td class="hdlist2">
<p>Based on ISematicModification (seemingly powerful but in fact weak) and IModification (less weak, but still very weak compared to ICompletionProposal) – only creates DocumentChanges.
+
Declarativ API that links to issue codes via annotations on 'fix' methods in AbstractDeclarativeQuickfixProvider.</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Best Practices
</td>
<td class="hdlist2">
<p>ICompletionProposal vs. DocumentChanges, ICompletionProposal is much more powerful. IModifications can also provide semantic changes, but not really recommended
+</p>
<div class="ulist">
<ul>
<li>
<p>Associated to isses via IssueCodes, @Fix similar to @Check API – only less powerful Xtext abstraction (no ICompletionProposal)</p>
</li>
<li>
<p>use issue data to provide hints for fix labels (which should be fast!) or solution strategies (but only strings) → do not compute the label for the fix from the model!</p>
</li>
<li>
<p>share code between checks and fixes → no built-in pattern, come up with utility methods (maybe define conventions)</p>
</li>
<li>
<p>maybe Sebastian can add a solution that more information is available via @Fix-approach</p>
</li>
<li>
<p>no order of quickfixes (sorted by name and priority, latter is not provided by default)</p>
</li>
<li>
<p>there can be several @Fix for a single issue code, or pass arbitrary number of resolution to the acceptor</p>
</li>
<li>
<p>for most cases simple Xtext quick fix api is good enough (e.g. all Xtend quick fixes use that)
+
→ Xtext feature request: solve multiple markers at a time (possible to do that right now: bind custom <code>XtextQuickAssistProcessor</code>, override <code>MarkerResolutionGenerator.getAdaptedResolutions(List&lt;IssueResolution&gt;</code>) – return a WorkbenchMarkerResolution)</p>
</li>
</ul>
</div>
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="sec:Quick_Assist"><a class="anchor" href="#sec:Quick_Assist"></a><a class="link" href="#sec:Quick_Assist">15.1.1.6. Quick Assist</a></h5>
<div class="quoteblock">
<blockquote>
"Quick assists perform local code transformations. They are invoked on a selection or a single cursor in the Java editor and use the same shortcut as quick fixes (Ctrl+1), but quick assist are usually hidden when an error is around. To show them even with errors present on the same line, press Ctrl+1 a second time." (Eclipse Help)
</blockquote>
</div>
<div class="paragraph">
<p><code>like a quickfix without a problem</code></p>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Examples
</td>
<td class="hdlist2">
<p>Add/remove inferred types</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Eclipse API
</td>
<td class="hdlist2">
<p>Takes cursor position</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Xtext
</td>
<td class="hdlist2">
<p>no Xtext support, e.g. no default implementation (XtextQuickAssistProcessor is a quick fix provider, has nothing to do with QuickAssist) but: XtextQuickAssistProcessor, override canAssist, override computeQuickAssistProposals</p>
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="sec:Clean_Up_Actions"><a class="anchor" href="#sec:Clean_Up_Actions"></a><a class="link" href="#sec:Clean_Up_Actions">15.1.1.7. Clean Up Actions</a></h5>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Examples
</td>
<td class="hdlist2">
<p>Remove unused local vars, sort members</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Eclipse API
</td>
<td class="hdlist2">
<p>None, JDT specific (see ICleanUp)</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Xtext Specifics
</td>
<td class="hdlist2">
<p>None</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Best Practice
</td>
<td class="hdlist2">
<p>Monkey sees - Monkey does (look at JDT), In the end a it’s a CompositeRefactoring, which is a CompletionProposal</p>
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="sec:Save_Actions"><a class="anchor" href="#sec:Save_Actions"></a><a class="link" href="#sec:Save_Actions">15.1.1.8. Save Actions</a></h5>
<div class="paragraph">
<p>Similar to clean up actions but performed on save</p>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Examples
</td>
<td class="hdlist2">
<p>Format on save, Organize imports on save</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Eclipse API
</td>
<td class="hdlist2">
<p>None, JDT specific (see IPostSaveListener)</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Xtext Specifics
</td>
<td class="hdlist2">
<p>None</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Best Practice
</td>
<td class="hdlist2">
<p>XtextDocumentProvider.doSaveDocument (maybe better solutions in the future ;-) )</p>
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="sec:Auto_Edit"><a class="anchor" href="#sec:Auto_Edit"></a><a class="link" href="#sec:Auto_Edit">15.1.1.9. Auto Edit</a></h5>
<div class="paragraph">
<p>Auto edit is about closing braces that just have been typed, adding indentation after a line break the code snippet <code>if (true)</code> so basically it should be unobtrusive typing aids.</p>
</div>
<div class="paragraph">
<p>By default, restore model structure when editing (guide the user to proper text formatting, help the parser). Should not be used for other purposes in order to not hinder the user’s flow of editing.</p>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Examples
</td>
<td class="hdlist2">
<p>( &#8594; ( &lt;cursor&gt; )</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Eclipse API
</td>
<td class="hdlist2">
<p>org.eclipse.jface.text.IAutoEditStrategy</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Xtext Specifics
</td>
<td class="hdlist2">
<p>org.eclipse.xtext.ui.editor.autoedit.AbstractEditStrategy, some utility methods + implements VerifyKeyListener. May use the ISourceViewer via implements ISourceViewerAware</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Best Practices
</td>
<td class="hdlist2">
<p>Keep it as it is.</p>
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>Fun example but not useful in practice cf. <a href="https://code.google.com/a/eclipselabs.org/p/xtext-forms-integration/source/browse/trunk/plugins/org.eclipse.xtext.example.domainmodel.ui/src/org/eclipse/xtext/example/ui/autoedit/FantasticAutoEditStrategy.java?r=19">FantasticAutoEditStrategy</a></p>
</div>
</div>
<div class="sect4">
<h5 id="sec:Template_Proposals"><a class="anchor" href="#sec:Template_Proposals"></a><a class="link" href="#sec:Template_Proposals">15.1.1.10. Template Proposals</a></h5>
<div class="paragraph">
<p>More sophisticated edit utils that are invoked by means of content assist.</p>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Examples
</td>
<td class="hdlist2">
<p>sysout &#8594; System.out.println(<code>&lt;cursor&gt;</code> );</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Eclipse API
</td>
<td class="hdlist2">
<p>Part of the completion proposal API, e.g. ICompletionProposal</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Xtext Specifics
</td>
<td class="hdlist2">
<p>org.eclipse.xtext.ui.editor.contentassist.ITemplateProposalProvider, template contexts along the grammar rules by default, need to be stripped down to become usable.</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Best Practice
</td>
<td class="hdlist2">
<p>ship some: create them manually in workbench, export them as XML, fix XML file (add IDs, in Xtext documentation), put XML file in folder <code>templates</code> in UI plugin where propose a certain proposal: customize XtextTemplateContextTypeRegistry (bind subclass, override register context types) – by default too many context types are registered placeholders inside templates specific to Xtext – RTFM</p>
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="sec:Outline_View___Quick_Outline"><a class="anchor" href="#sec:Outline_View___Quick_Outline"></a><a class="link" href="#sec:Outline_View___Quick_Outline">15.1.1.11. Outline View / Quick Outline</a></h5>
<div class="paragraph">
<p>Structural represenation of the file contents (usually with different filter and sorting strategies).</p>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Examples
</td>
<td class="hdlist2">
<p>Outline View (but not Navigator nor package explorer), Quick Outline (in Xtext: same provider)</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Eclipse API
</td>
<td class="hdlist2">
<p>org.eclipse.ui.views.contentoutline.IContentOutlinePage</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Xtext Specifics
</td>
<td class="hdlist2">
<p>Lazy tree creation, syncing via EObject ranges, thread save access to the EObject from nodes. Declarative API to create the tree contents. org.eclipse.xtext.ui.editor.outline.impl.DefaultOutlineTreeProvider
+
allow actions on outline nodes (e.g., goto referenced file in <code>import</code> of outline)</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Best Practice
</td>
<td class="hdlist2">
<div class="ulist">
<ul>
<li>
<p>Produced from semantic model, tree structure of outline nodes</p>
<div class="ulist">
<ul>
<li>
<p>show tree based on TypeModel, maybe filter out elements w/o SyntaxElements (with type model, this should be rather cheap!)</p>
</li>
<li>
<p>use icons and styled labels (first user impression!)</p>
</li>
</ul>
</div>
</li>
<li>
<p>May run in the background (BackgroundOutlineTreeProvider)</p>
</li>
<li>
<p>done lazily</p>
</li>
<li>
<p>workflow: reconceiler: outline is a model listener</p>
</li>
</ul>
</div>
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>Helpful tools for icons in outline view:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><a href="http://marketplace.eclipse.org/content/eclipse-icon-archive-tool">Eclipse view</a> to show available Eclipse icons (that are of course licenced under EPL) with possibility to export them (<a href="http://bwgz-org.googlecode.com/files/EclipseIconArchiveTool-1.pdf">documentation</a>)</p>
</li>
<li>
<p>overview of Eclipse icons: <a href="http://eclipse-icons.i24.cc/" class="bare">http://eclipse-icons.i24.cc/</a></p>
</li>
</ul>
</div>
</div>
<div class="sect4">
<h5 id="sec:Navigator__Package_Explorer__Project_Explorer"><a class="anchor" href="#sec:Navigator__Package_Explorer__Project_Explorer"></a><a class="link" href="#sec:Navigator__Package_Explorer__Project_Explorer">15.1.1.12. Navigator, Package Explorer, Project Explorer</a></h5>
<div class="paragraph">
<p>three <code>explorers</code> , Navigator <code>latest</code> and most extensible one</p>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Best Practices
</td>
<td class="hdlist2">
<p>use Navigator only! (RTFM, nothing specific to Xtext yet)</p>
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>cf. <a href="http://projects.eclipse.org/projects/technology.handly" class="bare">http://projects.eclipse.org/projects/technology.handly</a> <code>read index and show it in the navigator</code></p>
</div>
</div>
<div class="sect4">
<h5 id="sec:Hyperlinking_and_Navigation"><a class="anchor" href="#sec:Hyperlinking_and_Navigation"></a><a class="link" href="#sec:Hyperlinking_and_Navigation">15.1.1.13. Hyperlinking and Navigation</a></h5>
<div class="paragraph">
<p>Linking (propose multiple linking targets, e.g. goto declaration or goto implementation when CTRL (or other modifier) + Left Mouse Click on method when receiver type is interface - show all available implementations)</p>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Examples
</td>
<td class="hdlist2">
<p>Go to declaration, Go to implementation, Go to super</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Eclipse API
</td>
<td class="hdlist2">
<p>org.eclipse.jface.text.hyperlink.IHyperlinkDetector</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Xtext Specifics
</td>
<td class="hdlist2">
<p>org.eclipse.xtext.ui.editor.hyperlinking.DefaultHyperlinkDetector, navigation to EObject URI most interesting: SIGNIFICANT cf. org.eclipse.xtext.resource.ILocationInFileProviderExtension.RegionDescription</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Best Practice
</td>
<td class="hdlist2">
<div class="ulist">
<ul>
<li>
<p>subclass and bind IHyperlinkHelper (returns an array of possible links, first one is the default)</p>
</li>
<li>
<p>also see ILocationInFileProviderExtension (cf. navigation to syntax elements instead of types)</p>
</li>
</ul>
</div>
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="sec:Syntax_and_Semantic_Coloring"><a class="anchor" href="#sec:Syntax_and_Semantic_Coloring"></a><a class="link" href="#sec:Syntax_and_Semantic_Coloring">15.1.1.14. Syntax and Semantic Coloring</a></h5>
<div class="paragraph">
<p>Coloring based on the lexical tokens or based on the semantic tokens (the parsed model). The parser may treat certain lexical keywords as valid identifiers in some contexts. Some of those should not appear as keywords. Semantic coloring is usually more expensive to compute thus run in the background and with some delay</p>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Examples
</td>
<td class="hdlist2">
<p>Numbers, String literals (lexical) Escape sequences in Strings, method calls, property read / write access (semantic)</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Eclipse API
</td>
<td class="hdlist2">
<p><code>org.eclipse.jface.text.presentation.IPresentationDamager</code> <code>org.eclipse.jface.text.presentation.IPresentationRepairer</code> <code>org.eclipse.jface.text.rules.ITokenScanner</code>
+
Scan for tokens and associate text attributes with tokens. Compute the region of the document that has to be recolored after a text change. Tokens may not overlap.
+
Also Eclipse provides Themes that are styled via CSS. Coloring can be adjusted to themes where the logical names are mapped to different default values.</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Xtext Specifics
</td>
<td class="hdlist2">
<div class="ulist">
<ul>
<li>
<p><code>o.e.x.ui.editor.syntaxcoloring.ITextAttributeProvider</code> - associate Antlr token names with coloring styles (logical names of text coloring)</p>
</li>
<li>
<p><code>o.e.x.ui.editor.syntaxcoloring.AbstractAntlrTokenToAttributeIdMapper</code>- convert the antlr tokens to JFace ITokens with proper text applied</p>
</li>
<li>
<p><code>o.e.x.ui.editor.syntaxcoloring.IHighlightingConfiguration</code> - register logical text colorings with default values, yields a preference page and the proper configuration for the text attribute provider</p>
</li>
<li>
<p><code>o.e.x.ui.editor.syntaxcoloring.ISemanticHighlightingCalculator</code> - traverse the AST and associate arbitrary ranges of the text with.</p>
</li>
<li>
<p>logical coloring names (this is a key to a style stored in the preference store), if multiple styles are returned, styles will be merged if they overlap (and if possible); JFace constraints are implicitly fulfilled</p>
</li>
</ul>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
Best Practice
</td>
<td class="hdlist2">
<div class="ulist">
<ul>
<li>
<p>subclass DefaultSemanticHighlightingCalculator and bind ISemanticHighlightingCalculator</p>
<div class="ulist">
<ul>
<li>
<p>traverse resource from left to right (usually order of semantic elements – small performance improvement)</p>
</li>
</ul>
</div>
</li>
<li>
<p>provide new logical style: subclass DefaultHighlightingConfiguration and bind IHighlightingConfiguration; override configure (see overridden)</p>
</li>
<li>
<p>semantic coloring always wins</p>
</li>
<li>
<p>only a few decisions can me made in lexical coloring, override lexical:</p>
<div class="ulist">
<ul>
<li>
<p>subclass <code>DefaultAntlrTokenToAttributeIdMapper</code> bind <code>TokenTypeToStringMapper</code></p>
</li>
<li>
<p>e.g., color jsdoc comments differently to multiline, regex</p>
</li>
<li>
<p>e.g. color tags inside jsdocs or regex inside, use semantic coloring</p>
</li>
<li>
<p>lexical: different kind of keywords (e.g., N4JS keywords vs. JS keywords)</p>
</li>
</ul>
</div>
</li>
<li>
<p>change coloring (via toggle button), possible approach:</p>
<div class="ulist">
<ul>
<li>
<p>(inject singleton into highlighter, state of singleton is changed by toggle button, listen to that object in the editor, calculator cannot be triggered from outside due to UI-thread issues)</p>
</li>
<li>
<p>prefered: store state in preference store and get the information then from there in the hightligher, inject PreferencestoreAccess in Calculator</p>
</li>
</ul>
</div>
</li>
</ul>
</div>
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="sec:Code_Formatter"><a class="anchor" href="#sec:Code_Formatter"></a><a class="link" href="#sec:Code_Formatter">15.1.1.15. Code Formatter</a></h5>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Examples
</td>
<td class="hdlist2">
<p>Auto-Format Source Code, Auto-Format code inserted by code-rewrite</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Eclipse API
</td>
<td class="hdlist2">
<p><code>org.eclipse.jface.text.formatter.IContentFormatter</code> - here is the document and some range - modify at will</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Xtext Specifics
</td>
<td class="hdlist2">
<p>Declarative Formatting API (to be deprecated) - associate formatting rules with grammar elements New formatting API (mixture of declarative and imperative) - here is the model, do what you want (space before, linebreak after, indentation increase / decrease), the engine will merge your advices and apply them to the document</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Best Practice
</td>
<td class="hdlist2">
<p>wait for 2.8 (maybe in 2.7.x)</p>
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="sec:Wizards"><a class="anchor" href="#sec:Wizards"></a><a class="link" href="#sec:Wizards">15.1.1.16. Wizards</a></h5>
<div class="quoteblock">
<blockquote>
Wizards are used to guide the user through a sequenced set of tasks. Your plug-in can contribute wizards at predefined extension points in the workbench. It can also create and launch its own wizards.
</blockquote>
<div class="attribution">
&#8212; Eclipse Help
</div>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Examples
</td>
<td class="hdlist2">
<p>New N4JS Class</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Eclipse API
<br>
Xtext Specifics
</td>
<td class="hdlist2">
<div class="ulist">
<ul>
<li>
<p>Xtend based Wizards</p>
</li>
<li>
<p>also see Formular Editor for Embedded Xtext editor</p>
</li>
</ul>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
Best Practices
</td>
<td class="hdlist2">
<div class="ulist">
<ul>
<li>
<p>use preferences (could be hidden, so use them even if not made configurable to the user)</p>
</li>
<li>
<p>use standard JFace wizard API, use Xtend template expressions for file templates</p>
</li>
</ul>
</div>
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="sec:Cheat_Sheets"><a class="anchor" href="#sec:Cheat_Sheets"></a><a class="link" href="#sec:Cheat_Sheets">15.1.1.17. Cheat Sheets</a></h5>
<div class="quoteblock">
<blockquote>
<div class="paragraph">
<p>Composite cheat sheets provide guidance through complex problems by breaking the problem into a set of smaller tasks. Composite cheat sheets are registered using the the <code>org.eclipse.ui.cheatsheets.cheatSheetContent</code> extension point.</p>
</div>
</blockquote>
<div class="attribution">
&#8212; Eclipse Help
</div>
</div>
<div class="paragraph">
<p>(In Scala IDE: Work Sheets), often combined with Code Snippets</p>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Examples
</td>
<td class="hdlist2">
<p>Create Hello World Application</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Eclipse API
<br>
Xtext Specifics
</td>
<td class="hdlist2">
<p>None, probably the embedded editor could be used in a REPL (Read-Evaluate-Print-Loop)</p>
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="sec:Context_sensitive_Help"><a class="anchor" href="#sec:Context_sensitive_Help"></a><a class="link" href="#sec:Context_sensitive_Help">15.1.1.18. Context-sensitive Help</a></h5>
<div class="quoteblock">
<blockquote>
<div class="paragraph">
<p>A focused set of help topics that is related to the current context can be shown to users on demand using context-sensitive help. This form of user assistance is delivered to users when a platform-specific trigger is activated (e.g. F1 key on Windows, Ctrl+F1 on GTK, Help key on Carbon). Until Eclipse 3.1, context-sensitive help was presented in infopop windows. Since 3.1, a new Help view is the preferred way to deliver context-sensitive information to the user.</p>
</div>
</blockquote>
<div class="attribution">
&#8212; Eclipse Help
</div>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Examples
</td>
<td class="hdlist2">
<p>Help in Formular Editor, Help about syntax construct, API-Help</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Eclipse API
<br>
Xtext Specifics
</td>
<td class="hdlist2">
<p>None</p>
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="sec:Hovers"><a class="anchor" href="#sec:Hovers"></a><a class="link" href="#sec:Hovers">15.1.1.19. Hovers</a></h5>
<div class="paragraph">
<p>Hover allow to display additional information as soon as the cursor stays on a certain text region. Some hovers can be requested by shortcuts (e.g. F2) similar to sort of an online help.</p>
</div>
<div class="paragraph">
<p>Different kind of hovers may appear depending on the context, e.g. the error hover will have higher prio than the documentation hover. Different modifier keys may be assigned to request different hover kinds while hovering a region with the mouse. (didn’t a proper code pointer, though)</p>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Examples
</td>
<td class="hdlist2">
<p>Hover over method shows JSDoc, Signatures or inferred types, error / problem details</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Eclipse API
</td>
<td class="hdlist2">
<p><code>org.eclipse.jface.text.ITextHover</code> + <code>ITextHoverExtension*</code> - compute hover based on the region that is hovered. Various indirections with <code>IInformationControl</code> and <code>IInformationControlCreator</code> with many extension interfaces</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Xtext Specifics
</td>
<td class="hdlist2">
<p><code>org.eclipse.xtext.ui.editor.hover.IEObjectHover</code> - compute hover based on <code>EObjects</code></p>
</td>
</tr>
<tr>
<td class="hdlist1">
Best Practice
</td>
<td class="hdlist2">
<p>see XBase hover stuff</p>
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="sec:Folding"><a class="anchor" href="#sec:Folding"></a><a class="link" href="#sec:Folding">15.1.1.20. Folding</a></h5>
<div class="paragraph">
<p>Code folding allows to skip parts of the code that are mandatory semantically but usually do not provide added value for the reader, e.g. import sections</p>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Examples
</td>
<td class="hdlist2">
<p>Import section folding, folding of arbitrary methods or comments</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Eclipse API
</td>
<td class="hdlist2">
<p>Not much there, most of that stuff is implemented specific to JDT or ODE. Projections usually only work per line, that is, a subsection of a line cannot be folded, e.g. it’s not possible to show</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>var x = new Map&lt;String, List&lt;Pair&lt;String, Number&gt;&gt;&gt;()</code></pre>
</div>
</div>
<div class="paragraph">
<p>as</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>var x = new Map&lt;...&gt;()</code></pre>
</div>
</div>
<div class="paragraph">
<p>Line only limitation in SWT (a guess, didn’t work for Sebastian otherwise)</p>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
Xtext Specifics
</td>
<td class="hdlist2">
<div class="ulist">
<ul>
<li>
<p><code>org.eclipse.xtext.ui.editor.folding.DefaultFoldingRegionProvider</code> - here is the resource, compute the folding</p>
</li>
<li>
<p><code>org.eclipse.xtext.ui.editor.folding.DefaultFoldingStructureProvider</code> - bridge between editor and custom computation, preferences etc would be read from here</p>
</li>
<li>
<p>no preference page for folding provided by Xtext</p>
</li>
</ul>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
Best Practice
</td>
<td class="hdlist2">
<div class="ulist">
<ul>
<li>
<p>maybe limit to blocks (subclass default, bind to interface)</p>
</li>
<li>
<p>probably provide your own folding preference page</p>
</li>
</ul>
</div>
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="sec:Customizable_validation___severity"><a class="anchor" href="#sec:Customizable_validation___severity"></a><a class="link" href="#sec:Customizable_validation___severity">15.1.1.21. Customizable validation / severity</a></h5>
<div class="paragraph">
<p>Some problems are more important to the user than others so they want to change the severity.</p>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Examples
</td>
<td class="hdlist2">
<p>Deprecation could be an error, warning or ignored (e.g. in test projects)</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Eclipse API
</td>
<td class="hdlist2">
<p>None</p>
</td>
</tr>
<tr>
<td class="hdlist1">
Xtext Specifics
</td>
<td class="hdlist2">
<p>IssueSeverityProvider (since 2.6), Monkey sees monkey does: see subclasses of IssueSeverityProvider (we already do that)</p>
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="sec:Proposals"><a class="anchor" href="#sec:Proposals"></a><a class="link" href="#sec:Proposals">15.1.1.22. Proposals</a></h5>
<div class="paragraph">
<p>Created by Content Assist, Quick Fixes, Quick Assist.</p>
</div>
<div class="paragraph">
<p>Basics</p>
</div>
<div class="ulist">
<ul>
<li>
<p>simplest case: proposals are strings to be inserted</p>
</li>
<li>
<p>or displayed string is different from inserted one (e.g. FQN vs. simple)</p>
</li>
<li>
<p>ConfigurableCompletionProposal created via factory methods in AbstractN4JSProposalProvider (*create*Pro)</p>
</li>
<li>
<p>PrefixMatcher (by default CamelCase aware) – for filtering, it usually is not necessary to use it when computing the proposal (only if it expensive to compute proposals) – that is, prefix can be ignored when computing a proposal because the prefix matcher will filter out invalid proposals anyway</p>
</li>
<li>
<p>pass a filter (Guava preodicate) to filter out (semantically invalid) proposals, cf. lookupCrossReference(..) – for the things where there are proposals created by default</p>
</li>
<li>
<p>priority defined by an int – for sorting. Cf. ContentProposalPriorities → define default priorities (constant values) in N4JS, do not add some random integers!</p>
</li>
<li>
<p>modes: bind RepeatedContentAssistProcessor and enable modeaware in ProposalProvider (e.g. for private members which require a quickfix)</p>
</li>
<li>
<p>what could be done in the background: hover, lazy (not prepared) proposals (cf. JDT), Xtext 2.7.; different situations are processed in parallel</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Several changes (e.g. automatic import):</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>ConfigurableCompletionProposal.setTextApplier</code></p>
</li>
<li>
<p>TextApplier: can open dialogs etc., TextApplier is the callback</p>
</li>
<li>
<p>usual case: add text at cursor position and somewhere else:</p>
<div class="ulist">
<ul>
<li>
<p>get document in TextApplier</p>
</li>
<li>
<p>for performance, but also: do not use semantic changes in content assist, because model is broken (you will get funny things) – use model (AST) to get offset, but then insert line breaks etc. → maybe create utility class for retrieving current formattings which are then used in the text edit → maybe provide tools for retrieving certain locations (e.g. import section, field section, etc.)</p>
</li>
<li>
<p>do not create model (AST) fragments (which are then serialized), instead directly provide text</p>
</li>
<li>
<p>use TextEdit and MultiTextEdit</p>
</li>
<li>
<p>set TextViewer redraw to false and to true after the text edits were applied</p>
</li>
<li>
<p>have proper TESTS to ensure that file is not broken after the changes</p>
</li>
</ul>
</div>
</li>
<li>
<p><strong>LinkedEditing</strong>:</p>
<div class="ulist">
<ul>
<li>
<p>Linked-Editing mode in ConfigurableCompletionProposal with one editing group only (basically: move the cursor somewhere after editing it, see setSimpleLinkedMode)</p>
</li>
<li>
<p>do it manually: cf. LinkedPositionGroup (see call hierarchy of constructor) – used for quick fixes or refactorings rather for content assist</p>
</li>
</ul>
</div>
</li>
</ul>
</div>
</div>
</div>
<div class="sect3">
<h4 id="sec:Non_Eclipse_UI_Concepts"><a class="anchor" href="#sec:Non_Eclipse_UI_Concepts"></a><a class="link" href="#sec:Non_Eclipse_UI_Concepts">15.1.2. Non-Eclipse UI Concepts</a></h4>
<div class="paragraph">
<p>The following entries are not necessarily implemented yet.</p>
</div>
<div class="sect4">
<h5 id="sec:Overlays"><a class="anchor" href="#sec:Overlays"></a><a class="link" href="#sec:Overlays">15.1.2.1. Overlays</a></h5>
<div class="paragraph">
<p>An overlay is a small annotation similar to an hover, attached to a specific location in the editor and is moved with that location.</p>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Examples
</td>
<td class="hdlist2">
<p>Show inferred types</p>
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="sec:Goto__Inferred__Type"><a class="anchor" href="#sec:Goto__Inferred__Type"></a><a class="link" href="#sec:Goto__Inferred__Type">15.1.2.2. Goto (Inferred) Type</a></h5>
<div class="paragraph">
<p>Navigate to an inferred type (or other <code>invisible</code> information)</p>
</div>
</div>
<div class="sect4">
<h5 id="sec:Postfix_Completion"><a class="anchor" href="#sec:Postfix_Completion"></a><a class="link" href="#sec:Postfix_Completion">15.1.2.3. Postfix Completion</a></h5>
<div class="paragraph">
<p>(IntelliJ) Replace code <em>AFTER</em> an expression</p>
</div>
</div>
</div>
</div>
<div class="sect2">
<h3 id="_user-interface-resources"><a class="anchor" href="#_user-interface-resources"></a><a class="link" href="#_user-interface-resources">15.2. User Interface Resources</a></h3>
<div class="sect3">
<h4 id="_icons"><a class="anchor" href="#_icons"></a><a class="link" href="#_icons">15.2.1. Icons</a></h4>
<div class="paragraph">
<p>In parts, the N4JS IDE re-uses some of the icons that come with the Eclipse Platform as well as the Eclipse JDT development environment. However, in some cases we also provide our own icons to illustrate N4JS-specific concepts.</p>
</div>
<div class="sect4">
<h5 id="_eclipse-platform-icons"><a class="anchor" href="#_eclipse-platform-icons"></a><a class="link" href="#_eclipse-platform-icons">15.2.1.1. Eclipse Platform Icons</a></h5>
<div class="paragraph">
<p>When re-using the Eclipse Platform Icons, the icons are usually copied over to the <code>icons/</code> folder of the <code>org.eclipse.n4js.ui</code> bundle. In this folder, the <code>README.adoc</code> file keeps book on the origin of all the collected icons (e.g. different Eclipse Projects).</p>
</div>
</div>
<div class="sect4">
<h5 id="_n4js-specific-icons"><a class="anchor" href="#_n4js-specific-icons"></a><a class="link" href="#_n4js-specific-icons">15.2.1.2. N4JS Specific Icons</a></h5>
<div class="paragraph">
<p>In some cases, the icons the Eclipse eco-system provides do not suffice to sensibly express N4JS concepts. In these cases we provide our own icons. When designing those we try to imitate the general Eclipse artstyle in order for our icons to integrate well with the overall appearance of Eclipse.</p>
</div>
<div class="paragraph">
<p>For the creation of new icons, the <code>eclipse-svg-icons</code> repository (see <a href="https://github.com/Seung-Yoon/eclipse-svg-icons" class="bare">https://github.com/Seung-Yoon/eclipse-svg-icons</a>) has proven helpful. The repository contains raw SVG files which can be used to reproduce the bitmap icons that are contained in, for instance, the <code>org.eclipse.platform.ui</code> or <code>org.eclipse.jdt.ui</code> bundle. Based on that, common vector-graphics editing software may be used to further adapt color, form and style of existing icons (e.g. Inkscape <a href="https://inkscape.org/en/" class="bare">https://inkscape.org/en/</a>).</p>
</div>
</div>
<div class="sect4">
<h5 id="_high-resolution-icons"><a class="anchor" href="#_high-resolution-icons"></a><a class="link" href="#_high-resolution-icons">15.2.1.3. High Resolution Icons</a></h5>
<div class="paragraph">
<p>With the Neon release, Eclipse SWT introduced explicit support for high-DPI monitors (see <a href="https://www.eclipse.org/eclipse/news/4.6/platform.php#swt-autoscale" class="bare">https://www.eclipse.org/eclipse/news/4.6/platform.php#swt-autoscale</a>). In order to provide a good user experience, we want to provide high-DPI support for as many of our icons as possible. For that, it suffices to simply provide an alternative resource with higher resolution by appending the prefix @2x to its name (e.g. <code>class.png</code> and <code>class@2x.png</code>). Code-wise, no adjustments are required. In case of copied Eclipse Platform Icons, most of the time a corresponding 2x-version can be obtained from the original source. In case of N4JS Specific Icons, we export all icons in the resolutions 16x16 and 32x32. For that, it is of particular importance to make sure that the scaling is done in accordance with the native resolution (cf. pixel perfect scaling, also see <a href="https://en.wikipedia.org/wiki/Native_resolution" class="bare">https://en.wikipedia.org/wiki/Native_resolution</a>).</p>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_formatting"><a class="anchor" href="#_formatting"></a><a class="link" href="#_formatting">16. Formatting</a></h2>
<div class="sectionbody">
<div class="admonitionblock warning">
<table>
<tr>
<td class="icon">
<i class="fa icon-warning" title=""></i>
</td>
<td class="content">
This chapter may be outdated.
</td>
</tr>
</table>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:FmtObjective"><a class="anchor" href="#sec:FmtObjective"></a><a class="link" href="#sec:FmtObjective">16.1. Objective</a></h3>
<div class="paragraph">
<p>Writing textual code has many degrees of freedom. The resulting layout differs between authors. Carefully placing whitespace and newlines can increase the readability. Some handling of whitespace can be automated. This chapter describes the techniques used to automate the formatting to some degree.</p>
</div>
<div class="paragraph">
<p>Formatting N4js source code ensures a consistent style. It takes care of:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>surrounding language constructs with white space to improve readability</p>
</li>
<li>
<p>indenting logically grouped elements</p>
</li>
<li>
<p>wrapping long lines of code and comments</p>
</li>
<li>
<p>inserting semicolons, which would otherwise be automatically inserted (ASI)</p>
</li>
<li>
<p>formatting documentation</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Formatting will never alter the semantics of the code and it will not reorganize it.</p>
</div>
<div class="sect3">
<h4 id="sec:FmtFormatting_Comments"><a class="anchor" href="#sec:FmtFormatting_Comments"></a><a class="link" href="#sec:FmtFormatting_Comments">16.1.1. Formatting Comments</a></h4>
<div class="paragraph">
<p>N4js distinguishes five different types of comments <em>single line comments</em>, <em>not-indented single line comments</em>, <em>multiline comments</em>, <em>fixed multiline comments</em> and <em>Jsdoc style multiline comments.</em></p>
</div>
<div class="paragraph">
<p>Single line comments start with <code>//</code> and include all characters until the end of line. They usually will be indented and wrapped if they exceed the maximum line length unless they start immediately at the first column. Single line comments starting at position 0 are called <em>not-indented single line comments</em>.</p>
</div>
<div class="paragraph">
<p>Multiline comments start with <code>/*</code> and span all character including line breaks up the the end given by <code>*/</code> . The three variants are distinguished by the third and fourth character:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Comments starting with <code>/*-</code> are always fixed multiline comments.</p>
</li>
<li>
<p>Comments starting with <code>/**</code> and following any other character but <code>*</code> are Jsdoc style multiline comments. (E.g. <code>/**+</code> start Jsdoc but <code>/***</code> does not.)</p>
</li>
<li>
<p>All others starting with <code>/*</code> are ordinary multiline comments.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p><em>Not-indented single line comments</em> and <em>fixed multiline comments</em> will always remain as they are. Usually they are used to comment out code sections.</p>
</div>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:FmtArchitecture"><a class="anchor" href="#sec:FmtArchitecture"></a><a class="link" href="#sec:FmtArchitecture">16.2. Architecture</a></h3>
<div class="paragraph">
<p>Formatting mainly takes place in polymorphic dispatch methods <code>format</code> in class <code>N4JSFormatter</code>. Some language features are formatted with an entry-point using a super-class. This is mainly the case for structures that are sufficiently similar. E.g. <code>format( FunctionOrFieldAccessor )</code> is responsible for getter, setter, methods, functions, ….</p>
</div>
<div class="paragraph">
<p>Some common source-code formattings are grouped by <code>configureXY()</code> methods. They get called from the <code>format</code> methods for similar code structures, c.f. <code>configureAnnotations</code></p>
</div>
<div class="paragraph">
<p>Since we do not support formatting of the TypeExpression-language stand-alone, this class provides only one format-method throwing an <code>UnsupportedOperationException</code> type expressions formatting is defined in class <code>N4JSFormatter</code> at the end of the file.</p>
</div>
<div id="fig:formatter_overview" class="imageblock center">
<div class="content">
<img src="chapters/17_formatting/images/FormatterArchitecture.svg" alt="FormatterArchitecture">
</div>
<div class="title">Figure 47. Overview of classes used for formatting</div>
</div>
<div class="paragraph">
<p>The first entry-point for a script is <code>N4JSFormatter.format(Script, IFormattableDocument)</code>. Within this method an instance of <code>N4JSGenericFormatter</code> is created and used to configure general aspects of automatic-semicolon insertion (put a ’<code>;</code>’ where ASI took place in the parser) and handling of colons (’<code>:</code>’).</p>
</div>
<div class="sect3">
<h4 id="sec:Implementation_example"><a class="anchor" href="#sec:Implementation_example"></a><a class="link" href="#sec:Implementation_example">16.2.1. Implementation example</a></h4>
<div class="paragraph">
<p>Considering the following N4js-snippet where the return value of a function call will be casted to some type.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>functionCall("a","b") as MyType&lt;string&gt;</code></pre>
</div>
</div>
<div class="paragraph">
<p>The whole line is a CastExpression comprising of an expression (<code>functionCall(a,b)</code>) and a type reference (<code>MyType&lt;string&gt;</code>).</p>
</div>
<div class="paragraph">
<p>The <code>format</code>-dispatch method written in Xtend would look like:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code> def dispatch void format(CastExpression expr, extension IFormattableDocument document) {
expr.regionFor.keyword("as").prepend[newLines = 0; oneSpace].append[newLines = 0; oneSpace];
expr.expression.format;
expr.targetTypeRef.format;
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>In line 2 the format around the keyword <code>as</code> is specified where in line 3 and 4 the formatting of the containing elements will be dispatched.</p>
</div>
<div class="paragraph">
<p>Note that <code>regionFor</code> in line 2 is a method declared in <code>IFormattableDocument</code> and used via the extension-parameter <code>document</code>. It returns an object of type <code>ISemanticRegionFinder</code>. Invoking the <code>keyword</code> method on this object returns an instance of <code>ISemanticRegion</code> which will be passed to the extension methods <code>prepend</code> and <code>append</code> following the builder pattern. Both <code>prepend</code> and <code>append</code> take a lambda expression operating on a single parameter of type <code>IHiddenRegionFormatter</code>. Inside the lambda-expression this parameter is implicitly used to invoke the methods <code>setNewLines(0)</code> and <code>oneSpace()</code>. These calls simply disallow line-breaks around as and force the whitespace to be just a single character.</p>
</div>
<div class="paragraph">
<p>Possible other formatting instructions can be found in <code>IHiddenRegionFormatter</code>.</p>
</div>
<div class="paragraph">
<p>Due to some bugs in auto-wrapping<sup class="footnote">[<a id="_footnoteref_16" class="footnote" href="#_footnote_16" title="View footnote.">16</a>]</sup> unsuccessful attempts to wrap a line can insert unexpected new-lines in regions several lines in front of the currently treated source-line.</p>
</div>
<div class="paragraph">
<p>Debugging the formatter can be cumbersome as, due to GH-12<sup class="footnote">[<a id="_footnoteref_17" class="footnote" href="#_footnote_17" title="View footnote.">17</a>]</sup>, the <code>toString()</code> methods if internal data-structures throw exceptions.</p>
</div>
</div>
</div>
<div class="sect2">
<h3 id="sec:FmtFormatter_Implementation_Guidelines"><a class="anchor" href="#sec:FmtFormatter_Implementation_Guidelines"></a><a class="link" href="#sec:FmtFormatter_Implementation_Guidelines">16.3. Formatter Implementation Guidelines</a></h3>
<div class="ulist">
<ul>
<li>
<p>Each formatted element should only format it’s inner content. This avoids conflicting situations.</p>
</li>
<li>
<p>For each region possibly containing whitespace must be formatted.</p>
</li>
<li>
<p>Use priorities for conflict-resolutions sparse. For contradicting informations in the same region, the higher priority wins. If both information have the same priority, then an Exception will be thrown, showing two stack-traces to indicate the two code-regions being responsible for the situation.</p>
</li>
<li>
<p>For auto-wrapping a callback can be registered. In case of wrapping you can then conditionally change the format. Registering a callback implicitly sets the auto-wrap flag for the region.</p>
</li>
<li>
<p>Cover formatting with at least two different unit-tests. One having as little white-space as possible (all in one line) and the other as much white-space as possible in order to identify unformatted regions.</p>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="sec:FmtConfiguration"><a class="anchor" href="#sec:FmtConfiguration"></a><a class="link" href="#sec:FmtConfiguration">16.4. Configuration</a></h3>
<div class="paragraph">
<p>Some formattings can be customised through preference key-value pairs. Class <code>N4JSFormatterPreferenceKeys</code> acts as the entry-point to define such key-(default-)value pairs. Some preferences are inherited and are based on values stored in the default preference store (Line length, default tab width, …).</p>
</div>
<div class="paragraph">
<p>Currently the preferences are not yet accessible by the end-user.</p>
</div>
</div>
<div class="sect2">
<h3 id="sec:FmtUI_Integration"><a class="anchor" href="#sec:FmtUI_Integration"></a><a class="link" href="#sec:FmtUI_Integration">16.5. UI Integration</a></h3>
<div class="paragraph">
<p>Code formatting is invoked with standard key-strokes ( <kbd>CMD</kbd>+<kbd>Shift</kbd>+<kbd>F</kbd> on Mac, <kbd>Ctrl</kbd>+<kbd>Shift</kbd>+<kbd>F</kbd> on Windows)</p>
</div>
<div class="paragraph">
<p>There is no UI for preferences values yet.</p>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:FmtUnit_Testing_with_Xpect"><a class="anchor" href="#sec:FmtUnit_Testing_with_Xpect"></a><a class="link" href="#sec:FmtUnit_Testing_with_Xpect">16.6. Unit Testing with Xpect</a></h3>
<div class="paragraph">
<p>With Xpect Method <code>formattedLines</code> implemented in class <code>org.eclipse.n4js.xpect.FormatterXpectMethod</code> in bundle <code>org.eclipse.n4js.tests.helper</code> the formatting can be tested. The test method requires the number of lines which should be formatted. The desired test is given as a standard multiline expectation.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>/* XPECT formattedLines 1 ---
var a, b, c, d, e;
--- */
var a,b,c,d,e;</code></pre>
</div>
</div>
<div class="paragraph">
<p>Preferences can be configured in the Xpect setup section by providing string values. Numbers and booleans are converted automatically by the preferences framework.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>/* XPECT_SETUP org.eclipse.n4js.tests.N4JSXpectTest
ResourceSet {
ThisFile {}
File "wishesImported.n4js" { }
}
Preference "indentation" " " {}
Preference "line.width.max" "100" {}
Preference "format.auto_wrap_in_front_of_logical_operator" "false" {}
END_SETUP
*/</code></pre>
</div>
</div>
<div class="paragraph">
<p>Tip: Full coverage of the formatting can be tested via authoring the input using spaces as indentation characters if the formatter would use tabs or vice versa. That way untouched lines are distinguishable during the test-runs</p>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_external-libraries"><a class="anchor" href="#_external-libraries"></a><a class="link" href="#_external-libraries">17. External Libraries</a></h2>
<div class="sectionbody">
<div class="paragraph">
<p></p>
</div>
<div class="sidebarblock">
<div class="content">
<a href="https://github.com/eclipse/n4js/issues/1018" class="gray" title="GitHub Issues"><span class="image"><img src="images/issue.svg" alt=""></span> #1018</a>
<a href="https://github.com/eclipse/n4js/issues/397" class="gray" title="GitHub Issues"><span class="image"><img src="images/issue.svg" alt=""></span> #397</a>
<a href="https://github.com/eclipse/n4js/issues/809" class="gray" title="GitHub Issues"><span class="image"><img src="images/issue.svg" alt=""></span> #809</a>
<a href="https://github.com/eclipse/n4js/issues/714" class="gray" title="GitHub Issues"><span class="image"><img src="images/issue.svg" alt=""></span> #714</a>
<a href="https://github.com/eclipse/n4js/issues/653" class="gray" title="GitHub Issues"><span class="image"><img src="images/issue.svg" alt=""></span> #653</a>
<a href="https://github.com/eclipse/n4js/issues/862" class="gray" title="GitHub Issues"><span class="image"><img src="images/issue.svg" alt=""></span> #862</a>
<a href="https://github.com/eclipse/n4js/issues/1133" class="gray" title="GitHub Issues"><span class="image"><img src="images/issue.svg" alt=""></span> #1133</a>
</div>
</div>
<div class="paragraph">
<p><strong>External libraries</strong> are N4JS projects that are provided by the N4JS IDE:
the <em>built-in</em>/<em>shipped</em> libraries, and all <em>3rd-party libraries</em> that were installed by the <em>N4JS library manager</em>.
Each external library consist of a valid package.json file located in the project root and an arbitrary number of files supported by N4JS projects, e.g. <em>.n4js</em>, <em>.njsd</em> and <em>.js</em> files.
The purpose of the external libraries is to share and to provide core and third party functionality for N4JS developers both in compile and runtime without rebuilding them.</p>
</div>
<div class="paragraph">
<p><a href="#sec:Built-in_External_Libraries">[sec:Built-in_External_Libraries]</a> are external libraries that provide some basic functionality for N4JS programmers, such as the class <code>N4Injector</code>.</p>
</div>
<div class="paragraph">
<p><strong>3rd-party libraries</strong> are external libraries that are not built-in/shipped with the N4JS IDE.
Instead, they can be installed later by the user from third party providers.
Currently, only <em>npm packages</em> are supported.</p>
</div>
<div class="paragraph">
<p>The <strong>N4JS index</strong> is populated when the external libraries are compiled.
However, this compilation is only triggered through the library manager, but not when building workspace projects. (Self-evidently, the index is also populated when compiling workspace projects.)</p>
</div>
<div class="paragraph">
<p><strong>Name clashes</strong> of projects can happen and they are solved in the following order:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>User workspace projects always shadow external libraries.</p>
</li>
<li>
<p>In case of a name clash between a shipped and a 3rd-party library, the 3rd-party library shadows the shipped project.</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>The <strong>N4JS library manager</strong> is a tool in the N4JS IDE to view and manage external libraries.
In particular, the user can (un-)install new 3rd-party libraries, or can trigger the build of all external libraries to re-populate the N4JS index.
The library manager also supports other maintenance actions such as deleting all 3rd-party libraries.</p>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Major_Components"><a class="anchor" href="#sec:Major_Components"></a><a class="link" href="#sec:Major_Components">17.1. Major Components</a></h3>
<div class="paragraph">
<p>External libraries are supported based on different components all over the application.</p>
</div>
<div class="paragraph">
<p>The followings are the most important ones:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><strong>External Resources</strong> (<code>IExternalResource</code>)</p>
<div class="ulist">
<ul>
<li>
<p>These are customized <code>IResource</code> implementations for external projects, folders and files.</p>
</li>
<li>
<p>With this approach the <code>IProject</code>, <code>IFolder</code> and <code>IFile</code> interfaces have been implemented. Each implementation is backed by a pure <code>java.io.File</code> based resource.</p>
</li>
<li>
<p>When accessing such external resources for example visiting purposes, getting the members of the resource or simply deleting the resource, internally each requests will be directly performed on the wrapped <code>java.io.File</code> without accessing the <code>org.eclipse.core.resources.IWorkspace</code> instance.</p>
</li>
</ul>
</div>
</li>
<li>
<p><strong>External Library Workspace</strong></p>
<div class="ulist">
<ul>
<li>
<p>This is a kind of dedicated workspace for external libraries and their dependencies.</p>
</li>
<li>
<p>Any query requests to retrieve a particular project or any dependencies of a particular project via the <code>IN4JSCore</code> singleton service will delegated to its wrapped <code>N4JSModel</code> singleton. Internally the <code>N4JSModel</code> has a reference to a workspace for all the ordinary workspace projects and another reference to the workspace for external libraries. Each query requests will be forwarded to the workspace for the ordinary projects first, and then to the external library workspace. If ordinary project workspace can provide any meaningful response for a request, then the external library workspace will not be accessed at all. Otherwise the query will be executed against the external library workspace. This fallback mechanism provides a pragmatic solution to the project shadowing feature. The project shadowing will be described in details later in this section.</p>
</li>
<li>
<p>The <strong>External Library Workspace</strong> is only supported and available in the IDE case, in the headless case there are no external libraries available from this dedicated workspace. Since the Xtext index creation and the entire build infrastructure is different, it is supported via target platform file. This is described in more details in a later section (<a href="#sec:Headless_External_Library_Support">Headless External Library Support</a>]).</p>
</li>
</ul>
</div>
</li>
<li>
<p><strong>External Library Preference Store</strong></p>
<div class="ulist">
<ul>
<li>
<p>This preference store is being used to register and un-register external library root folders into its underlying ordered list. A folder is called as an external library root folder if it is neither equal with the Eclipse workspace root nor being nested in the workspace root and contains zero to any external libraries.</p>
</li>
<li>
<p>Whenever any modifications are being saved in this preference store the <em>External Library Workspace</em> will be updated as well, new libraries will be registered into the workspace and removed libraries will be cleaned up from the workspace.</p>
</li>
<li>
<p>When the N4JS IDE application is started in production mode, the initial state of the preference store is being pre-populated with default values. This is necessary to provide built-in libraries to end users. These default values and additional advanced configurations will be mentioned in more details later in this section.</p>
</li>
</ul>
</div>
</li>
<li>
<p><strong>Library Manager</strong></p>
<div class="ulist">
<ul>
<li>
<p>This service is responsible for downloading and installing third party <em>npm</em> packages into the <code>node_modules</code> folder of the N4JS IDE. After downloading, the newly-installed and/or updated packages are registered as external libraries into the system.</p>
</li>
</ul>
</div>
</li>
<li>
<p><strong>External Library Builder</strong></p>
<div class="ulist">
<ul>
<li>
<p>This service is responsible for updating the persistent Xtext index with the currently available external libraries.</p>
</li>
<li>
<p>Unlike in case of any other ordinary projects, this builder does not triggers a build via the <code>org.eclipse.core.internal.events.BuildManager</code> but modifies the persisted Xtext index (<code>IBuilderState</code>) directly.</p>
</li>
<li>
<p>Considers shadowed external libraries when updating the persisted Xtext index.</p>
</li>
<li>
<p>Makes sure that the external library related Xtext index is persistent and will be available on the next application startup.</p>
</li>
</ul>
</div>
</li>
<li>
<p><strong>External Library Xtext Index Persister</strong></p>
<div class="ulist">
<ul>
<li>
<p>This class is responsible for recovering the consistent external library Xtext index state at application startup.</p>
</li>
<li>
<p>Scheduled on the very first application startup to prepare the Xtext index for the available external libraries.</p>
</li>
<li>
<p>Recovers the Xtext index state after a force quit and/or application crash.</p>
</li>
</ul>
</div>
</li>
<li>
<p><strong>External Library Preference Page</strong></p>
<div class="ulist">
<ul>
<li>
<p>Preference page to configure and update the state of the <em>External Library Preference Store</em>.</p>
</li>
<li>
<p>Provides a way to install <em>npm</em> dependencies as external libraries into the application.</p>
</li>
<li>
<p>Reloads the external libraries. Gets the most recent state of N4JS type definition files and updates the Xtext index content based on the current state of the external libraries.</p>
</li>
<li>
<p>Exports the current npm dependency configuration as a target platform file. This will be discussed in another section ([sec:Headless_External_Library_Support]).</p>
</li>
</ul>
</div>
</li>
<li>
<p><strong>Miscellaneous UI Features</strong></p>
<div class="ulist">
<ul>
<li>
<p>Searching for types provided by external libraries.</p>
</li>
<li>
<p>Opening external modules in read-only editor.</p>
</li>
<li>
<p>Navigation between external types.</p>
</li>
<li>
<p><em>Project Explorer</em> contribution for showing external dependencies for ordinary workspace projects.</p>
</li>
<li>
<p>Editor-navigator linking support for external modules.</p>
</li>
<li>
<p>Installing third party npm dependencies directly from package.json editor via a quick fix.</p>
</li>
</ul>
</div>
</li>
</ul>
</div>
<div class="sect3">
<h4 id="subsec:External_Resources"><a class="anchor" href="#subsec:External_Resources"></a><a class="link" href="#subsec:External_Resources">17.1.1. External Resources</a></h4>
<div class="paragraph">
<p>This approach provides a very pragmatic and simple solution to support external libraries in both in the <code>IN4JSCore</code> and in the <code>IBuilderState</code>. While <code>IN4JSCore</code> supports a completely transparent way of external libraries via the <code>IN4JSProject</code> interface all over in the application, the <code>IBuilderState</code> is responsible for keeping the Xtext index content up to date with the external libraries. Below picture depicts the hierarchy between the ordinary <code>IResource</code> and the <code>IExternalResource</code> instances. As described above each external resource is backed by a <code>java.io.File</code> resource and each access and operation being invoked on the <code>IResource</code> interface will be delegated to this backing resource.</p>
</div>
<div id="fig:External_Resources_Hierarchy" class="imageblock center">
<div class="content">
<img src="chapters/20_externalLibraries/images/externalResources.svg" alt="externalResources">
</div>
<div class="title">Figure 48. External Resources Hierarchy</div>
</div>
</div>
<div class="sect3">
<h4 id="subsec:External_Library_Workspace"><a class="anchor" href="#subsec:External_Library_Workspace"></a><a class="link" href="#subsec:External_Library_Workspace">17.1.2. External Library Workspace</a></h4>
<div class="paragraph">
<p>External library workspace is an extension of the <code>InternalN4JSWorkspace</code>. This workspace is used for storing and managing external libraries all over the application. External libraries can be registered into the workspace by providing one to many external library root folder locations. The provided root folder locations will be visited in an ordered fashion and the contained external libraries (N4JS projects) will be registered into the application. If an external library from a root folder has been registered, then a forthcoming occurrence of an external library with the same artefact identifier (and same folder name) will be ignored at all. For instance let assume two external library root locations are available <code>ER1</code> and <code>ER2</code>, also <code>ER1</code> contains <code>P1</code> and <code>P2</code> external libraries, while <code>ER2</code> contains <code>P2</code> and <code>P3</code>. After registering the two roots into the workspace <code>ER1</code> will be processed first, and <code>P1</code> and <code>P2</code> will be registered to the workspace, when processing the forthcoming <code>ER2</code> root, <code>P2</code> will be ignored at all as an external with the same name exists. Finally <code>P3</code> will be registered to the workspace. External libraries cannot be registered directly into the workspace it is done automatically by the <em>External Library Preference Store</em> and by the <em>npm Manager</em>.</p>
</div>
</div>
<div class="sect3">
<h4 id="subsec:External_Library_Preference_Store"><a class="anchor" href="#subsec:External_Library_Preference_Store"></a><a class="link" href="#subsec:External_Library_Preference_Store">17.1.3. External Library Preference Store</a></h4>
<div class="paragraph">
<p>This persistent cache is used for storing an ordered enumeration of registered external library root folder locations. Whenever its internal state is being persisted after a modification, all registered modification listeners will be synchronously notified about this change. All listeners will receive the store itself with the updated state. There are a couple of registered listeners all over the application listening to store update events but the most important one is the <em>External Library Workspace</em> itself. After receiving an external library preference store update event, the external library workspace will calculate the changes from its own state: creates a sort of difference by identifying added, removed and modified external libraries. Also tracks external library root location order changes. Once the workspace has calculated the changes<sup class="footnote">[<a id="_footnoteref_18" class="footnote" href="#_footnote_18" title="View footnote.">18</a>]</sup> it will interact with the <em>External Library Builder Helper</em> which will eventually update the persisted Xtext index directly through the <code>IBuilderState</code>. After the Xtext index content update all ordinary workspace projects that directly depend either on a built or a cleaned external library will be automatically rebuilt by the external library workspace.</p>
</div>
</div>
<div class="sect3">
<h4 id="subsec:npm_Manager"><a class="anchor" href="#subsec:npm_Manager"></a><a class="link" href="#subsec:npm_Manager">17.1.4. Library Manager</a></h4>
<div class="paragraph">
<p>This service is responsible for downloading, installing third party npm dependencies into the local file system. This is done directly by <code>npm</code> from <code>Node.js</code>. Once an npm package has been downloaded and installed it will be registered into the external library workspace. As part of the registration, the Xtext index content will be updated and all dependent ordinary workspace projects will be rebuilt automatically. An npm package cannot be installed via the <em>Library Manager</em> if it already installed previously.</p>
</div>
</div>
<div class="sect3">
<h4 id="subsec:External_Library_Builder_Helper"><a class="anchor" href="#subsec:External_Library_Builder_Helper"></a><a class="link" href="#subsec:External_Library_Builder_Helper">17.1.5. External Library Builder</a></h4>
<div class="paragraph">
<p>This builder is responsible for updating the persisted Xtext index state with external library content directly through the <code>IBuilderState</code>. When providing a subset of external libraries to either build or clean, internally it orders the provided external libraries based on the project dependencies. Also, it might skip building all those external libraries that have are being shadowed by a workspace counterpart. An external library is being shadowed by an ordinary workspace project, if the workspace project is accessible and has exactly the same project name as the external library.</p>
</div>
</div>
<div class="sect3">
<h4 id="subsec:External_Library_Xtext_Index_Persister"><a class="anchor" href="#subsec:External_Library_Xtext_Index_Persister"></a><a class="link" href="#subsec:External_Library_Xtext_Index_Persister">17.1.6. External Library Xtext Index Persister</a></h4>
<div class="paragraph">
<p>By default Xtext provides a way to fix corrupted index or to recreate it from scratch in case of its absence. Such inconsistent index states could occur due to application crashes or due to non-graceful application shutdowns. Although this default recovery mechanism provided by Xtext works properly, it is provided only for projects that are available in the Eclipse based workspace (<code>org.eclipse.core.resources.IWorkspace</code>) but non of the external libraries are not available from the Eclipse based workspace, so inconsistent external library index content cannot be recovered by this default mechanism. N4JS IDE contributes its own logic to recover index state of external N4JS libraries. When the default Xtext index recovery runs, then it will trigger a external reload as well. This external reload is guaranteed to run always after the default recovery mechanism.</p>
</div>
</div>
<div class="sect3">
<h4 id="subsec:External_Library_Preference_Page"><a class="anchor" href="#subsec:External_Library_Preference_Page"></a><a class="link" href="#subsec:External_Library_Preference_Page">17.1.7. External Library Preference Page</a></h4>
<div class="paragraph">
<p>This preference page provides a way to configure the external libraries by adding and removing external library root folders, also allows the user to reorder the configured external library root locations. Besides that, npm packages can be installed into the application as external libraries. Neither removing nor reordering built-in external libraries are supported, hence these operations are disabled for built-ins on the preference page. No modifications will take effect unless the changes are persisted with the <code>Apply</code> button. One can reset the configurations to the default state by clicking on the <code>Restore Defaults</code> button then on the <code>Apply</code> button. The <code>Reload</code> button will check whether new type definition files are available for npm dependencies, then reloads the persistent Xtext index content based on the available external libraries. Once the external library reloading has been successfully finished, all dependent workspace projects will be rebuilt as well. From the preference page one can export the installed and used third party npm packages as a target platform. This exported target platform file can be used with the headless compiler. After setting up the headless compiler with this exported target platform file, the headless tool will collect and download all required third party npm dependencies.</p>
</div>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Headless_External_Library_Support"><a class="anchor" href="#sec:Headless_External_Library_Support"></a><a class="link" href="#sec:Headless_External_Library_Support">17.2. Headless External Library Support</a></h3>
<div class="paragraph">
<p>The headless compiler is not capable of supporting built-in libraries. The whole build and Xtext index creation infrastructure is different in the IDE and in the headless case. Also, due to its archive nature (<code>n4jsc.jar</code>) of the headless tool, neither the runtime nor the <code>Mangelhaft</code> libraries can be loaded into the headless compiler.</p>
</div>
<div class="paragraph">
<p>The headless compiler supports downloading, installing and using third party <code>npm</code> packages. To enable this feature one has to configure the target platform via the <code>–targetPlatformFile</code> (or simply <code>-tp</code>) and the <code>–targetPlatformInstallLocation</code> (or simply <code>-tl</code>) arguments.</p>
</div>
<div class="paragraph">
<p>If the target platform file argument is configured, then all third party dependencies declared in the target platform file will be downloaded, installed and made available for all the N4JS projects before the compile (and run) phase. If the target platform file is given but the target platform install location is not specified (via the <code>–targetPlatformInstallLocation</code> argument), then a the compilation phase will be aborted and the execution will be interrupted.</p>
</div>
<div class="paragraph">
<p>For more convenient continuous integration and testing purposes there are a couple of additional exception cases with respect to the the target platform file and location that users of the headless compiler have to keep in mind. These are the followings:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>–targetPlatformSkipInstall</code>. Usually dependencies defined in the target platform file will be installed into the folder defined by option <code>–targetPlatformInstallLocation</code>. If this flag is provided, this installation will be skipped, assuming the given folder already contains the required files and everything is up-to-date. Users have to use this flag with care, because no checks will be performed whether the location actually contains all required dependencies.</p>
</li>
<li>
<p>If <code>–targetPlatformSkipInstall</code> is provided the <code>–targetPlatformInstallLocation</code> parameter is completely ignored.</p>
</li>
<li>
<p>If <code>–targetPlatformSkipInstall</code> is provided the <code>–targetPlatformFile</code> parameter is completely ignored.</p>
</li>
<li>
<p>If neither <code>–targetPlatformInstallLocation</code> not <code>–targetPlatformFile</code> parameters are specified the headless tool will treat this case as an implicit <code>–targetPlatformSkipInstall</code> configuration.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>If the target platform install location is configured, and the target platform file is given as well, then all third party dependencies specified in the target platform file will be downloaded to that given location. If the target platform file is given, but the target platform install location is not specified, then a the compilation phase will be aborted and the execution will be interrupted.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-bash" data-lang="bash">java -jar n4jsc.jar -projectlocations /path/to/the/workspace/root -t allprojects -tp /absolute/path/to/the/file -tl /path/to/the/target/platform/install/location -rw nodejs -r moduleToRun</code></pre>
</div>
</div>
<div class="sect3">
<h4 id="_custom-npm-settings"><a class="anchor" href="#_custom-npm-settings"></a><a class="link" href="#_custom-npm-settings">17.2.1. Custom npm settings</a></h4>
<div class="paragraph">
<p>In some cases there is a need for custom npm settings, e.g. custom npm registry. Those kind of configurations are
supported via <code>.npmrc</code> file (see <a href="https://docs.npmjs.com/files/npmrc" class="bare">https://docs.npmjs.com/files/npmrc</a>).</p>
</div>
<div class="paragraph">
<p>In N4JSIDE user can specify path to his custom configuration file in the preference page.</p>
</div>
<div class="paragraph">
<p>For the commandline N4JSC.jar provides special option <code>-npmrcRootLocation</code> that allows headless compiler to
use custom settings.</p>
</div>
</div>
</div>
<div class="sect2">
<h3 id="sec:lmFutureWork"><a class="anchor" href="#sec:lmFutureWork"></a><a class="link" href="#sec:lmFutureWork">17.3. Future Work</a></h3>
<div class="paragraph">
<p>Some aspects not covered in current design, but worth consideration in the future</p>
</div>
<div class="sect3">
<h4 id="subsec:lmMultipleDependencyScope"><a class="anchor" href="#subsec:lmMultipleDependencyScope"></a><a class="link" href="#subsec:lmMultipleDependencyScope">17.3.1. Multiple Dependency Scope</a></h4>
<div class="paragraph">
<p>npm scope dependencies</p>
</div>
<div class="dlist">
<dl>
<dt class="hdlist1"><strong>DEPENDENCY_DEVELOPMENT</strong> </dt>
<dd>
<p><a href="https://docs.npmjs.com/files/package.json#devdependencies" class="bare">https://docs.npmjs.com/files/package.json#devdependencies</a></p>
</dd>
<dt class="hdlist1"><strong>DEPENDENCY_PEER</strong> </dt>
<dd>
<p><a href="https://docs.npmjs.com/files/package.json#peerdependencies" class="bare">https://docs.npmjs.com/files/package.json#peerdependencies</a></p>
</dd>
<dt class="hdlist1"><strong>DEPENDENCY_BUNDLE</strong> </dt>
<dd>
<p><a href="https://docs.npmjs.com/files/package.json#bundleddependencies" class="bare">https://docs.npmjs.com/files/package.json#bundleddependencies</a></p>
</dd>
<dt class="hdlist1"><strong>DEPENDENCY_OPTIONAL</strong> </dt>
<dd>
<p><a href="https://docs.npmjs.com/files/package.json#optionaldependencies" class="bare">https://docs.npmjs.com/files/package.json#optionaldependencies</a></p>
</dd>
<dt class="hdlist1"><strong>DEPENDENCY_PROVIDES</strong> </dt>
<dd>
<p><a href="http://www.rpm.org/wiki/PackagerDocs/Dependencies#Provides" class="bare">http://www.rpm.org/wiki/PackagerDocs/Dependencies#Provides</a></p>
</dd>
<dt class="hdlist1"><strong>DEPENDENCY_WEAK</strong> </dt>
<dd>
<p><a href="http://www.rpm.org/wiki/PackagerDocs/Dependencies#Weakdependencies" class="bare">http://www.rpm.org/wiki/PackagerDocs/Dependencies#Weakdependencies</a></p>
</dd>
</dl>
</div>
</div>
<div class="sect3">
<h4 id="subsec:lmRunTestsFromLibrary"><a class="anchor" href="#subsec:lmRunTestsFromLibrary"></a><a class="link" href="#subsec:lmRunTestsFromLibrary">17.3.2. Run Tests from TestLibrary</a></h4>
<div class="paragraph">
<p>Imagine we are implementing some API, and we want to run tests for that API. Tests are delivered to us as separate package, and there is not direct association between implementation and test projects (tests are not depending on implementation). Still we want to run provided tests to see if our implementation complies with API tests, e.g. AcceptanceTest suite for Application written against application sdk.</p>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="sec:JSON_Support"><a class="anchor" href="#sec:JSON_Support"></a><a class="link" href="#sec:JSON_Support">18. JSON Support</a></h2>
<div class="sectionbody">
<div class="admonitionblock warning">
<table>
<tr>
<td class="icon">
<i class="fa icon-warning" title=""></i>
</td>
<td class="content">
This chapter may be outdated.
</td>
</tr>
</table>
</div>
<div class="sect2">
<h3 id="sec:JSON_Parser"><a class="anchor" href="#sec:JSON_Parser"></a><a class="link" href="#sec:JSON_Parser">18.1. JSON Parser</a></h3>
<div class="paragraph">
<p>For the JavaScript Object Notation format, a multitude of specifications exist (e.g. [<a href="#RFC8259">RFC8259</a>], [<a href="#ECMA404">ECMA404</a>], [<a href="#RFC7158">RFC7158</a>]). While all specifications agree on the basic structure of JSON, many special cases exist, which remain un-addressed by the specifications. Therefore, in practice many parsers exhibit implementation-specific behavior. See [<a href="#Ser18">Ser18</a>] for a discussion of many of these cases.</p>
</div>
<div class="paragraph">
<p>The initial grammar of the N4JS JSON parser is mostly based on [<a href="#ECMA404">ECMA404</a>] while further adaptions have been made to accommodate for special cases such as escaping unicode control characters in string literals. This section discusses the different aspect in which our parser exhibits special behavior in order to parse a maximum of real-world JSON text. If applicable, we will also discuss differences between JSON and the N4JS syntax for object literals.</p>
</div>
<div class="sect3">
<h4 id="sec:JSON_Parser_Unicode_Escaping"><a class="anchor" href="#sec:JSON_Parser_Unicode_Escaping"></a><a class="link" href="#sec:JSON_Parser_Unicode_Escaping">18.1.1. Escaping Unicode Control Characters in String Literals</a></h4>
<div class="paragraph">
<p>In comparison to ECMAScript (and thus N4JS), JSON only requires the escaping of unicode control characters in the range from <code>U+0000</code> to <code>U+001F</code> when used in a string literal. This includes the line termination characters <code>U+000A</code> or <code>\n</code> and <code>U+000D</code> or <code>\r</code>. However, different from ECMAScript, JSON allows the unescaped use of the unicode characters <code>U+2028</code> and <code>U+2029</code> within string literals. Therefore, the JSON parser differs from the behavior of the N4JS parser.</p>
</div>
</div>
<div class="sect3">
<h4 id="sec:JSON_Parser_Empty_Text"><a class="anchor" href="#sec:JSON_Parser_Empty_Text"></a><a class="link" href="#sec:JSON_Parser_Empty_Text">18.1.2. Empty Text</a></h4>
<div class="paragraph">
<p>While the abovementioned JSON specifications do not allow for empty JSON text (e.g. no data, only whitespace data), our parser is tolerant towards such inputs. The reason behind this decision is that it allows users of the N4JS IDE, to create new empty JSON files without experiencing any parser errors.</p>
</div>
</div>
<div class="sect3">
<h4 id="sec:JSON_Parser_Nested_Structures"><a class="anchor" href="#sec:JSON_Parser_Nested_Structures"></a><a class="link" href="#sec:JSON_Parser_Nested_Structures">18.1.3. Nested Structures</a></h4>
<div class="paragraph">
<p>Since the N4JS JSON parser is implemented in terms of a recursive decent parser, the parsable inputs are limited in terms of nesting. This results in a stack overflow exception for highly nested input data.</p>
</div>
</div>
<div class="sect3">
<h4 id="sec:JSON_Parser_Whitespace"><a class="anchor" href="#sec:JSON_Parser_Whitespace"></a><a class="link" href="#sec:JSON_Parser_Whitespace">18.1.4. Whitespace</a></h4>
<div class="paragraph">
<p>[<a href="#RFC7158">RFC7158</a>] and [<a href="#ECMA404">ECMA404</a>] both define whitespace characters of JSON to be exclusively <code>U+0009</code>, <code>U+000A</code>, <code>U+000D</code> and <code>U+0020</code>. However, for now we adopt the whitespace strategy of the N4JS parser, which allows for additional whitespace characters (also see [<a href="#ECMA15a">ECMA15a</a>] section 7.2 White Space). This only applies to JSON text outside of string literals.</p>
</div>
</div>
<div class="sect3">
<h4 id="sec:JSON_Parser_Comments"><a class="anchor" href="#sec:JSON_Parser_Comments"></a><a class="link" href="#sec:JSON_Parser_Comments">18.1.5. Comments</a></h4>
<div class="paragraph">
<p>Although JSON as a standard does not specify any notion of source code comments, we allow them on a parser level. (single line comments introduced by <code>//</code> and multi-line comments using <code>/*</code> and <code>*/</code>). After parsing, we issue corresponding validation messages that indicate to the user, that such comments will possibly not be parsable in a different context (e.g. by npm in case of package.json files).</p>
</div>
</div>
</div>
<div class="sect2">
<h3 id="sec:JSON_Language_Extensions"><a class="anchor" href="#sec:JSON_Language_Extensions"></a><a class="link" href="#sec:JSON_Language_Extensions">18.2. JSON Language Extensions</a></h3>
<div class="paragraph">
<p>Generally, our JSON support was implemented with generic JSON support in mind. However, for some types of JSON files (e.g. <code>package.json</code> files) we provide custom behavior to better assist the user. This includes custom JSON validators and resource description strategies that only apply to certain types of JSON files. In order to keep our JSON implementation independent from N4JS-specific code, these concerns are separated using an Eclipse Extension Point. In the headless case, extension also need to be registered manually via the <code>JSONExtensionRegistry</code>. In the following we present the different aspect in which the JSON language can be extended.</p>
</div>
<div class="sect3">
<h4 id="sec:JSON_Validator_Extensions"><a class="anchor" href="#sec:JSON_Validator_Extensions"></a><a class="link" href="#sec:JSON_Validator_Extensions">18.2.1. JSON Validator Extensions</a></h4>
<div class="paragraph">
<p>Via the JSON validator extensions (cf. <code>IJSONValidatorExtension</code>), other bundles can register JSON validator extensions that introduce domain-specific validation for specific types of JSON files.</p>
</div>
<div class="sect4">
<h5 id="sec:File_Specitic_Validator_Extensions"><a class="anchor" href="#sec:File_Specitic_Validator_Extensions"></a><a class="link" href="#sec:File_Specitic_Validator_Extensions">18.2.1.1. File-Specific Validator Extensions</a></h5>
<div class="paragraph">
<p>For every JSON resource that is validated, all registered JSON validator extensions are executed. For a validator extension to be specific to a certain type of file, one may override the Xtext method <code>isResponsible</code> which checks whether a validator applies on a per resource basis.</p>
</div>
</div>
<div class="sect4">
<h5 id="sec:JSON_Declarative_JSON_Validator_Extensions"><a class="anchor" href="#sec:JSON_Declarative_JSON_Validator_Extensions"></a><a class="link" href="#sec:JSON_Declarative_JSON_Validator_Extensions">18.2.1.2. Declarative JSON Validator Extensions</a></h5>
<div class="paragraph">
<p>In addition to Xtext&#8217;s <code>@Check</code> methods, we provide an annotation <code>@CheckProperty</code> that allows to declare JSON-specific check methods that only apply to certain property paths of a JSON document. The <code>@CheckProperty</code> annotation can only be used when inheriting from the <code>AbstractJSONValidatorExtension</code> class. The following example outlines its usage:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-java" data-lang="java">(...)
@CheckProperty(propertyPath = "prop1")
public void checkProperty(JSONValue propertyValue) {
// validate JSONValue for top-level property 'prop1'
}
@CheckProperty(propertyPath = "nested.prop2")
public void checkNestedProperty(JSONValue propertyValue) {
// validate JSONValue for the nested property 'nested.prop2'
}
(...)</code></pre>
</div>
</div>
<div class="paragraph">
<p>The examples illustrates how property-check-method can be declared. In a JSON document the first check method will only be invoked for the JSON value that is set for top-level property <code>prop1</code>. The second check method is invoked for the nested property <code>prop2</code> of the object that is set for the top-level property <code>prop2</code>.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-json" data-lang="json">{
"prop1": 1, // checkProperty applies
"nested": {
"prop2": 2 // checkNestedProperty applies
}
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>When inheriting from the <code>AbstractJSONValidatorExtension</code>, the usual <code>addIssue</code> methods may be used in order to issue validation messages. Furthermore, the issue codes that are being used for the messages may originate from the bundle that hosts the validator extension.</p>
</div>
</div>
</div>
<div class="sect3">
<h4 id="_json-resource-description-strategy"><a class="anchor" href="#_json-resource-description-strategy"></a><a class="link" href="#_json-resource-description-strategy">18.2.2. JSON Resource Description Strategy</a></h4>
<div class="paragraph">
<p>Via JSON resource description extensions (cf. <code>IJSONResourceDescriptionExtension</code>), other bundles can define a custom resource description strategy for specific types of JSON files. As a consequence, these extensions define what information on the contents of a JSON file is stored in the Xtext index (cf. <code>IResourceDescriptions</code>).</p>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_jsdoc"><a class="anchor" href="#_jsdoc"></a><a class="link" href="#_jsdoc">19. JSDoc</a></h2>
<div class="sectionbody">
<div class="admonitionblock warning">
<table>
<tr>
<td class="icon">
<i class="fa icon-warning" title=""></i>
</td>
<td class="content">
This chapter may be outdated.
</td>
</tr>
</table>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Design_Rationale"><a class="anchor" href="#sec:Design_Rationale"></a><a class="link" href="#sec:Design_Rationale">19.1. Design Rationale</a></h3>
<div class="paragraph">
<p>JSDoc parser provides general API for parsing and obtaining information embedded in comments. This may have low significance for N4JS itself but is essential when working with JS dialects (e.g. vanilla JS) that use JSDoc comments for enrich given dialect with semantic information. Anticipated uses may include: - type information extraction when importing external code - validation of links between commented fragments - supporting markup for documentation</p>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title=""></i>
</td>
<td class="content">
Although current focus is on migration process we want to provide general solution that can be customized for given use cases
</td>
</tr>
</table>
</div>
<div class="sect3">
<h4 id="_general-design"><a class="anchor" href="#_general-design"></a><a class="link" href="#_general-design">19.1.1. General Design</a></h4>
<div class="paragraph">
<p>When using the API client has to create instance of DocletParser and configure it with <em>LienTag</em>s and <em>InLineTag</em>s that are to be used.</p>
</div>
<div class="paragraph">
<p>LineTags can depend on InlineTags (e.g. InlineTg in description of parameters of the LineTag). API provides support for that but client has to configure this.</p>
</div>
<div id="fig:cd_JSDocParserAPI" class="imageblock center">
<div class="content">
<img src="chapters/23_jsdoc/images/cd_JSDocParserAPI.svg" alt="cd JSDocParserAPI">
</div>
<div class="title">Figure 49. JSDoc Parser API (without model)</div>
</div>
<div class="paragraph">
<p>Initiated DocletParser can be used to parse <em>String</em> containing given JSDoc comment.Based on provided <em>ITagDefinition</em>s parser will parse input string and return JSDoc DOM AST (see fig. XX with ecore diagram). By querying tree structure client can obtain information extracted form parsed input String.</p>
</div>
<div id="fig:cd_JSDocModel" class="imageblock center">
<div class="content">
<img src="chapters/23_jsdoc/images/cd_JSDocModel.svg" alt="cd JSDocModel">
</div>
<div class="title">Figure 50. JSDoc AST/DOM Model</div>
</div>
<div class="paragraph">
<p>The root of the this AST is the Doclet. The doclet itself contains some content (as it is a Composite containing ContentNodes), usually Description and an arbitrary number of LineTags. LineTags are created by custom ITagDefinition implementations that extend AbstractLineTagDefinition. They contain the title and an arbitrary number of values, stored in a map. These values are Composite nodes as well, containing tag specific content. E.g., the parameter tag will create the values for the parameter type, the parameter name, and the description. Description (in LineTags as well as in doclet itself) is free text with optional InlineTags. InlineTags are created by custom ITagDefinition that extends AbstractInlineTagDefinition. In general tag values are designed as a comprise between a very general tree (basically containing only text nodes) and a structured typed tree (containing type expressions etc.). Although it should be possible to model a tag value by only using Text nodes, some special typed nodes are provided for sake of simplicity (and probably performance). E.g., TypeReferences are modeled using a typed node, in order to simplify external handling of these references (for type analysis, and refactorings such as renaming a type). New typed nodes will probably be introduced (see below). Markers can be attached to nodes for internal purposes. E.g., a marker can be used to distinguish different syntax versions of defining an array (<code>String[]</code> vs. <code>Array&lt;String&gt;</code>).Literals can be used to simplify rewriting. That is, they can contain information on line breaks, JSDoc line prefixes etc., which are not needed for the semantics of a tag, but only for the syntax (not used yet). Also, each node contains a position for simplifying rewriting.</p>
</div>
<div id="fig:cd_JSDocParserAndModel" class="imageblock center">
<div class="content">
<img src="chapters/23_jsdoc/images/cd_JSDocParserAndModel.svg" alt="cd JSDocParserAndModel">
</div>
<div class="title">Figure 51. JSDoc Parser and Model</div>
</div>
</div>
<div class="sect3">
<h4 id="sec:Type_Expressions"><a class="anchor" href="#sec:Type_Expressions"></a><a class="link" href="#sec:Type_Expressions">19.1.2. Type Expressions</a></h4>
<div class="paragraph">
<p>Type expressions are are not handled by JSDoc Parser by itself as instances of the grammars are use case specific, e.g. for migration purposes grammar for type expressions used in migration process is specific to the in question therefore specific type expressions parser will be provided separately.DocLetParser by default can only extract String representing given type expression, parsing and interpreting it stays in responsibility of given tag implementation provided by the client.</p>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_docexporter"><a class="anchor" href="#_docexporter"></a><a class="link" href="#_docexporter">20. DocExporter</a></h2>
<div class="sectionbody">
<div class="admonitionblock warning">
<table>
<tr>
<td class="icon">
<i class="fa icon-warning" title=""></i>
</td>
<td class="content">
This chapter may be outdated.
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>The DocExporter exports JavaDoc from source files to adoc files.
In particular it combines information about methods with tests in order to create specification documents.</p>
</div>
<div class="sect2">
<h3 id="sec:Specification_Exporter"><a class="anchor" href="#sec:Specification_Exporter"></a><a class="link" href="#sec:Specification_Exporter">20.1. Specification Exporter</a></h3>
<div class="paragraph">
<p>The specification exporter creates and merges artifacts.
<a href="#fig:api_test_spec">Fig. API Test Spec</a> sketches the relation between API (i.e., n4jsd files with classifiers),
tests (i.e., N4JS test classes and methods), and specification (Documentation with JSDOC markers).</p>
</div>
<div id="fig:api_test_spec" class="imageblock center">
<div class="content">
<img src="chapters/24_docexporter/images/api_test_spec.svg" alt="api test spec">
</div>
<div class="title">Figure 52. Component/Class pseudo diagram: Relation API, Tests and Specification</div>
</div>
<div class="paragraph">
<p><a href="#fig:cd_jsdocreader">Fig. Exporter</a> shows the classes that read and analyze Java source documentation.</p>
</div>
<div id="fig:cd_jsdocreader" class="imageblock center">
<div class="content">
<img src="chapters/24_docexporter/images/cd_jsdocreader.svg" alt="cd jsdocreader">
</div>
<div class="title">Figure 53. Java reader classes</div>
</div>
<div class="paragraph">
<p><a href="#fig:cd_adocexporter">Fig. Exporter</a> shows the content classes of the exporter that contain the generated documentation contents.</p>
</div>
<div id="fig:cd_adocexporter" class="imageblock center">
<div class="content">
<img src="chapters/24_docexporter/images/cd_adocexporter.svg" alt="cd adocexporter">
</div>
<div class="title">Figure 54. Exporter content classes</div>
</div>
<div class="paragraph">
<p><a href="#fig:cd_docexporter_model">Fig. DocExporter Model</a> shows the model, which may be used for other doc exporters as well.</p>
</div>
<div id="fig:cd_docexporter_model" class="imageblock center">
<div class="content">
<img src="chapters/24_docexporter/images/cd_docexporter_model.svg" alt="cd docexporter model">
</div>
<div class="title">Figure 55. Exporter model classes</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_rename-refactoring"><a class="anchor" href="#_rename-refactoring"></a><a class="link" href="#_rename-refactoring">21. Rename Refactoring</a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>The rename refactoring operation is implemented based on current Xtext&#8217;s rename refactoring implementation. However, lots of customization have been done in order to make Rename Refactoring work for N4JS. In order to understand N4JS customization, it is imperative to understand how Xtext implements rename refactoring. In this chapter, we will focus on Xtext&#8217;s architecture for rename refactoring. Additionally, we will point to the components that have been customized for N4JS.</p>
</div>
<div class="sect2">
<h3 id="_rename-refactoring-ui-interaction"><a class="anchor" href="#_rename-refactoring-ui-interaction"></a><a class="link" href="#_rename-refactoring-ui-interaction">21.1. Rename Refactoring UI interaction</a></h3>
<div class="paragraph">
<p>Xtext&#8217;s implementation allows rename refactoring be in either one of two modes (1) Direct refactoring mode (3) Refactoring with dialog mode. Diagram <a href="#fig:rename_refactoring_communication_diagram_part1">Direct Rename Refactoring UI interaction</a> shows the UI interaction in <em>direct refactoring mode</em>.</p>
</div>
<div id="fig:rename_refactoring_communication_diagram_part1" class="imageblock center">
<div class="content">
<img src="chapters/25_renameRefactoring/images/rename_refactoring_communication_diagram_part1.svg" alt="title-"Direct Rename Refactoring UI interaction"">
</div>
</div>
<div class="paragraph">
<p>In this diagram, the classes in yellow are customized by N4JS implementation to handle N4JS-specific characteristics.</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>DefaultRenameElementHandler</code>: Our custom N4JS implementation converts the selected element to be renamed into its corresponding TModule element.</p>
</li>
<li>
<p><code>ILinkedPositionGroupCalculator</code>: This class is responsible for calculating locations of names to be changed during linked edit mode. We need to provide a custom N4JS implementation to handle composed elements.</p>
</li>
<li>
<p><code>RenameElementProcessor</code>: We need to provide a custom N4JS implementation to add N4JS-specific validation of conditions, e.g. checking name conflicts etc.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>The key class for creating updates of a declaration and its associated references is <code>RenameElementProcessor</code>. In the following section, we will see how this class interacts with other classes to achieve this.</p>
</div>
</div>
<div class="sect2">
<h3 id="_renameelementprocessor-interaction"><a class="anchor" href="#_renameelementprocessor-interaction"></a><a class="link" href="#_renameelementprocessor-interaction">21.2. RenameElementProcessor interaction</a></h3>
<div class="paragraph">
<p>Diagram <a href="#fig:rename_refactoring_communication_diagram_part2">RenameElementProcessor interaction</a> shows the interaction of <code>RenameElementProcessor</code> and other classes to create changes for both declaration and references during rename refactoring.</p>
</div>
<div id="fig:rename_refactoring_communication_diagram_part2" class="imageblock center">
<div class="content">
<img src="chapters/25_renameRefactoring/images/rename_refactoring_communication_diagram_part2.svg" alt="title-"RenameElementProcessor interaction"">
</div>
</div>
<div class="paragraph">
<p>As seen in the diagram, there are two stages of creating updates:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Creating updates for declaration is done by <code>IRenameStrategy</code> and</p>
</li>
<li>
<p>Creating updates for references is done by <code>ReferenceUpdateDispatcher</code>. <code>ReferenceUpdateDispatcher</code> in turn delegates the finding of references to <code>IReferenceFinder</code>.</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>The text edits for changing the definition and the references are accumulated by an <code>IRefactoringUpdateAcceptor</code>.</p>
</div>
<div class="paragraph">
<p>The classes in yellow are customized by N4JS implementation.</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>IRenameStrategy</code>: the custom N4JS implementation creates updates for constituent members of composed elements.</p>
</li>
<li>
<p><code>IReferenceFinder</code>: the custom N4JS implementation used for finding references of a declaration.</p>
</li>
<li>
<p><code>RefactoringCrossReferenceSerializer</code>: custom N4JS implementation to retrieve the updated name for cross references. For some unknown reason, the default implementation does not work correctly.</p>
</li>
</ul>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="chap:flowgraphs"><a class="anchor" href="#chap:flowgraphs"></a><a class="link" href="#chap:flowgraphs">22. Flow Graphs</a></h2>
<div class="sectionbody">
<div class="sect2 language-n4js">
<h3 id="sec:flowgraphs_overview"><a class="anchor" href="#sec:flowgraphs_overview"></a><a class="link" href="#sec:flowgraphs_overview">22.1. Flow graphs overview</a></h3>
<div class="paragraph">
<p>In this chapter, the control and data flow analyses are introduced.
Since not all AST elements are relevant for the control or data flow analyses, a new marker class is introduced called <code>ControlFlowElement</code>.
All AST elements which are part of the control flow graph implement this class.
The term control flow is abbreviated as <em>CF</em> and hence <code>ControlFlowElement</code>s are abbreviated as <em>CF elements</em>.</p>
</div>
<div class="paragraph">
<p></p>
</div>
<div class="sidebarblock">
<div class="content">
<a href="https://github.com/eclipse/n4js/issues/120" class="gray" title="GitHub Issues"><span class="image"><img src="images/issue.svg" alt=""></span> #120</a>
</div>
</div>
<div class="paragraph">
<p>The following picture shows the control flow graph of the function <code>f</code>.</p>
</div>
<div class="imageblock">
<div class="content">
<img src="chapters/30_flowgraphs/images/cfg_for_loop.png" alt="cfg for loop">
</div>
<div class="title">Figure 56. Control flow graph of a simple function</div>
</div>
<div class="sect3">
<h4 id="_internal-graph"><a class="anchor" href="#_internal-graph"></a><a class="link" href="#_internal-graph">22.1.1. Internal graph</a></h4>
<div class="paragraph">
<p>Every <em>internal graph</em> refers to a single <em>control flow container</em>.
The graph consists of <em>nodes</em> and <em>edges</em>, where the edges are instances of <code>ControlFlowEdge</code>, and nodes are instances of <code>Node</code>.
Additionally, a so called <em>complex node</em> is used to group nodes that belong to the same CF element.</p>
</div>
<div class="dlist">
<dl>
<dt class="hdlist1">Internal graph</dt>
<dd>
<p>Control flow graphs are created based on the AST elements.
Nevertheless, a fine-grained abstraction is used that is called <em>internal graph</em>.
The internal graph reflects all control flows and data effects that happen implicitly and are part of the language&#8217;s semantics.
For instance, the for-of statement on iterable objects forks the control flow after invoking the <code>next()</code> method.
This is done implicitly and not part of the written source code.
Moreover, this invocation could cause side effects.
These control flows and effects are reflected using the internal graph.
To implement analyses that refer to AST elements, an API for flow analyses is provided which hides the internal graph and works with AST elements only.
In the following, the term <em>control flow graph</em> refers to the internal graph.</p>
</dd>
<dt class="hdlist1">Control flow container</dt>
<dd>
<p>At several places in the AST, an execution of statements or elements can happen.
Obviously, statements can be executed in bodies of methods or function expressions.
In addition, execution can also happen in field initializers or the <code>Script</code> itself.
Since all these AST elements can contain executable control flow elements (CF elements), they thus contain a control flow graph.
In the following, these AST elements are called <em>control flow containers</em> or CF containers.</p>
</dd>
<dt class="hdlist1">Nodes</dt>
<dd>
<p>Nodes represent complete CF elements or parts of them.
For instance, simple CF elements like a <code>break</code> statement are represented using only one node.
Regarding more complex CF elements that introduce a more complex control flow, due to e.g. nested expressions, several nodes represent one CF element.
All nodes of a single CF element are grouped within a complex node.</p>
</dd>
<dt class="hdlist1">Edges</dt>
<dd>
<p>Edges reference a start and an end node which reflects a forward control flow direction, which is in the following called forward traverse direction.
Traversing from end to start of an edge is thus called backward traverse direction.
The so called <em>next node</em> is either the end node in context of forward, or the start node in context of backward traverse direction.
Edges also reflect the reason of the control flow using a control flow type.
The default control flow type is called <code>Successor</code> and such edges connect two ordinary subsequent nodes.
Other types like <code>Return</code> or <code>Break</code> indicate control flow that happens due to return or break statements.
A special control flow type is <code>Repeat</code> that indicates the entry of a loop body.
This edge is treated specially when traversing the control flow graph to avoid infinitive traversals of loops.</p>
</dd>
<dt class="hdlist1">Complex node</dt>
<dd>
<p>The complex node always has a single entry and exit node, no matter the control flow it causes.
For instance, although the for-statement can contain various control flows among its nested CF elements, its complex node still has a single entry and exit node.
This simplifies concatenating subsequent complex nodes, since only their exit nodes have to be connected to the following entry node.
Aside from exit and entry node, a complex node usually contains additionally nodes to represent the CF element.
These nodes are connected by control flow edge so that their control flow lies within complex node.
However, regarding nested CF elements, the control flow leaves and re-enters a complex node.
To specify which CF element is nested, a delegating node (<code>DelegatingNode</code>) is created that points to the nested CF element.</p>
</dd>
</dl>
</div>
<div class="paragraph">
<p>Consider that source code elements can be nested like expressions that have sub-expressions as in <code>1 + 2 * 3</code>.
Also statements can contain other statements like in <code>if (true) return;</code>.
The design of the control flow graph deals with this nesting by mapping CF elements to several nodes.
All nodes of one CF element are aggregated into the <em>complex node</em>.</p>
</div>
<div id="img:internalGraph" class="imageblock">
<div class="content">
<img src="chapters/30_flowgraphs/images/internalGraph.png" alt="internalGraph">
</div>
<div class="title">Figure 57. The source code of <code>1+2</code> creates an internal graph of three complex nodes to deal with nested integer literals</div>
</div>
<div class="paragraph">
<p>The example in the <a href="#img:internalGraph">figure above</a> shows the internal graph produced by the source code <code>1+2</code>.
Additionally, a simpler version of the internal graph is shown (called <em>User Graph View</em>), with which client analyses deal.
The user graph view is only a view on the internal graph, but does not exist as an own instance.
In the figure, the nesting of the integer literals becomes obvious:
The control flow edges of delegating nodes targets entry nodes of different CF elements.
Also, there are CF edges from the exit nodes of these nested CF elements to return the control flow.</p>
</div>
<div id="img:cn_for_stmt" class="imageblock">
<div class="content">
<img src="chapters/30_flowgraphs/images/cn_for_stmt.png" alt="cn for stmt">
</div>
<div class="title">Figure 58. Complex Node of for statement</div>
</div>
<div class="paragraph">
<p>In the <a href="#img:cn_for_stmt">above figure</a>, the complex node of the for statement is shown.
The details of the complex nodes of the nested CF elements (such as the initializer or the body statement) are omitted.
The figure displays the control flow fork after the condition and also shows the <em>Repeat</em> edge that targets the for body.
The node called <em>Catch Node</em> is used in situations when there are jumping control flows introduced for instance by a continue statement.
The catch will will then be the target of an control flow edge that starts at the continue statement.</p>
</div>
<div class="paragraph">
<p>The graph of nodes and edges is constructed in the following order.</p>
</div>
<div class="ulist">
<ul>
<li>
<p>First, for every CF element a complex node and all its nodes are created.
Also, the nodes within the complex node are connected according to their control flow behavior.</p>
</li>
<li>
<p>Second, all subsequent complex nodes are connected by connecting their exit and entry nodes.
Moreover, nested complex nodes are connected by interpreting the delegating nodes.</p>
</li>
<li>
<p>Third, jumping control due to <code>return</code>, <code>throw</code>, <code>break</code> or <code>continue</code> statements is computed.
This is done by first deleting the successor edge of the jumping statement and introducing a new control flow edge that ends at the jump target.</p>
</li>
</ul>
</div>
</div>
<div class="sect3">
<h4 id="_optimizations"><a class="anchor" href="#_optimizations"></a><a class="link" href="#_optimizations">22.1.2. Optimizations</a></h4>
<div class="paragraph">
<p>The internal graph contains many nodes to simplify the graph construction.
However, these nodes carry no relevant information when traversing the graph.
Consequently, in an optimization step, they are removed from the graph for performance reasons.</p>
</div>
<div class="paragraph">
<p>A node removal for a node <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>n</mi><mn>2</mn></msub></math> is done by replacing the path <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>n</mi><mn>1</mn></msub><mo>-</mo><mo>&gt;</mo><msub><mi>n</mi><mn>2</mn></msub><mo>-</mo><mo>&gt;</mo><msub><mi>n</mi><mn>3</mn></msub></math> by the new path <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>n</mi><mn>1</mn></msub><mo>-</mo><mo>&gt;</mo><msub><mi>n</mi><mn>3</mn></msub></math>.
These removals are done for delegating nodes that only have one incoming and one outgoing edge.</p>
</div>
<div class="paragraph">
<p>A second kind but similar optimization reduces the number of helper nodes that are used as entry nodes.
In case a complex nodes consists only of exactly one entry and one exit node, both of these nodes are collapsed into one node.
This remaining node then is the representing node of the AST element.</p>
</div>
</div>
<div class="sect3">
<h4 id="_api-for-client-analyses"><a class="anchor" href="#_api-for-client-analyses"></a><a class="link" href="#_api-for-client-analyses">22.1.3. API for client analyses</a></h4>
<div class="paragraph">
<p>To implement client analyses based on the control flow graph, the three classes <code>GraphVisitor</code>, <code>GraphExplorer</code> and <code>BranchWalker</code> are provided.
They provide the means to visit CF elements in a control flow graph and also to traverse single control flow paths.
The method <code>N4JSFlowAnalyses#analyze</code> can execute several client analyses in one run to maintain scalability.</p>
</div>
<div class="sect4">
<h5 id="_mapping-from-internal-to-ast-elements"><a class="anchor" href="#_mapping-from-internal-to-ast-elements"></a><a class="link" href="#_mapping-from-internal-to-ast-elements">22.1.3.1. Mapping from internal to AST elements</a></h5>
<div class="paragraph">
<p>The API classes work with AST elements such as <code>ControlFlowElement</code> instead of the internally used graph classes <code>ComplexNode</code>, <code>Node</code> or <code>ControlFlowEdge</code>.
The mapping from internal classes to AST elements is done in the <code>GraphVisitor</code> class.</p>
</div>
<div class="paragraph">
<p>Note that the control flow graph has the following properties:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>ExpressionStatement</code>s are not represented.
Instead, only their expressions are represented.
Nevertheless, the API can deal with calls that refer to expression statements, e.g. when requesting their successors.</p>
</li>
<li>
<p>Control statements are also not represented in the graph, but can also be used in calls to the API.
The reason is, that it is unclear when a control statement (e.g. a for loop) is visited exactly.</p>
</li>
<li>
<p>Since a <code>FlowEdge</code> which connects two <code>ControlFlowElement</code>s can represent multiple internal edges, it can have multiple <code>ControlFlowType</code>s.</p>
</li>
</ul>
</div>
</div>
<div class="sect4">
<h5 id="_graph-visitor"><a class="anchor" href="#_graph-visitor"></a><a class="link" href="#_graph-visitor">22.1.3.2. Graph visitor</a></h5>
<div class="paragraph">
<p>Graph visitors traverse the control flow graphs of every CF container of a script instance in the following two traverse directions:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><em>Forward</em>: from the container&#8217;s start to all reachable CF graph elements.</p>
</li>
<li>
<p><em>Backward</em>: from the container&#8217;s end to all reachable CF graph elements.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>In each traverse direction, the graph visitor visits every reachable CF element and edge.
Note that neither empty statements nor control statements are part of the control flow graph.
The order of visited CF elements is related to either a breadth or a depth search on the CF graph.
However, no specific order assumptions are guaranteed.</p>
</div>
<div class="imageblock">
<div class="content">
<img src="chapters/30_flowgraphs/images/deadcode.png" alt="deadcode">
</div>
<div class="title">Figure 59. The CF elements <code>"loop"</code> and <code>"end"</code> are dead code and displayed in grey.</div>
</div>
</div>
<div class="sect4">
<h5 id="_graph-explorer"><a class="anchor" href="#_graph-explorer"></a><a class="link" href="#_graph-explorer">22.1.3.3. Graph explorer</a></h5>
<div class="paragraph">
<p>Graph visitors can request a <em>graph explorer</em> to be activated under specific conditions related to the client analysis.
A graph explorer is the start point to analyze control flow branches.
The first control flow branches is started directly at the graph explorer&#8217;s creation site, but of course this first branches might fork eventually.
The graph explorer keeps track of all forked branches that originate at its activation site.
It also provides the means to join previously forked branches again.</p>
</div>
</div>
<div class="sect4">
<h5 id="_branch-walker"><a class="anchor" href="#_branch-walker"></a><a class="link" href="#_branch-walker">22.1.3.4. Branch walker</a></h5>
<div class="paragraph">
<p>With every graph explorer, a branch walker is created that traverses the control flow graph beginning from the activation site of the graph explorer.
On every such branch, the two visit methods of CF elements and edges respectively, are called in the order of the traverse direction.
Every time the branch forks, the fork method of the branch walker is invoked and creates another branch walker which will continue the traversal on the forked branch.
The fork method can be used to copy some path data or state to the newly forked branch walker.
Note that every edge is always followed by the branch walker except for repeat edges which are followed exactly twice.
The reason to follow them twice is that first, following them only once would hide those control flows that re-visit the same CF elements due to the loop.
Second, following them more than twice does not reveal more insights, but only increases the number of branches.
When control flow branches merge again, for instance at the end of an <code>if</code>-statement, two or more branch walkers are merged into a new succeeding one.
The graph explorer provides the means to do this.
In case a CF element has no next elements, the branch walker terminates.</p>
</div>
</div>
<div class="sect4">
<h5 id="_example-1-compute-string-for-each-path"><a class="anchor" href="#_example-1-compute-string-for-each-path"></a><a class="link" href="#_example-1-compute-string-for-each-path">22.1.3.5. Example 1: Compute string for each path</a></h5>
<div class="paragraph">
<p>Let&#8217;s assume that we want to compute all control flow branches of a function and use the client API for that.
The function <code>f()</code> in the following code snippet has four control flow branches: <code>1 &#8594; 2</code>, <code>&#8594; 3 &#8594;</code>, <code>&#8594; 4 &#8594;</code> and <code>5</code>.</p>
</div>
<div class="listingblock">
<div class="title">Function <code>f()</code> has four control flow branches.</div>
<div class="content">
<pre class="highlight"><code>function f() {
1;
if (2)
3;
else
4;
5;
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>To compute these control flow branches, the class <code>AllBranchPrintVisitor</code> extends the <code>GraphVisitor</code>.
Already in the method <code>initializeMode()</code> a graph explorer is activated.
Note that the method <code>requestActivation()</code> can be understood as a <code>addListener</code> method for a listener that listens to visit events on nodes and edges.
Immediately after the activation request, the first branch walker is created in the method <code>firstBranchWalker()</code>.</p>
</div>
<div class="paragraph">
<p>The first visited CF element of the branch walker will then be the expression <code>1</code>.
It is formed into a string and added to the variable <code>curString</code>.
After expression <code>1</code>, the flow edge from <code>1</code> to <code>2</code> is visited.
This will concatenate the string <code>&#8594;</code> to the path string.
Variable <code>curString</code> will eventually hold the branch string like <code>1 &#8594; 2</code>.
Since the control flow forks after <code>2</code>, the method <code>forkPath()</code> is called and creates two new instances of a branch walker.
These new instances succeed the the first branch walker instance and each traverses one of the branches of the <code>if</code>-statement.
When the <code>if</code>-statement is passed, these two branches are merged into a new succeeding branch walker.
After all branch walkers are terminated, the graph explorer and graph visitor are also terminated.
The method <code>getBranchStrings()</code> collects all four computed strings from the variable <code>curString</code> of all branch walkers.</p>
</div>
<div class="listingblock">
<div class="title">Implementation of a graph visitor that computes all control flow paths</div>
<div class="content">
<pre class="highlight"><code>class AllBranchPrintVisitor extends GraphVisitor {
protected void initializeMode(Mode curDirection, ControlFlowElement curContainer) {
super.requestActivation(new AllBranchPrintExplorer());
}
class AllBranchPrintExplorer extends GraphExplorer {
class AllBranchPrintWalker extends BranchWalker {
String curString = "";
protected void visit(ControlFlowElement cfe) {
curString += cfe.toString();
}
protected void visit(FlowEdge edge) {
curString += " -&gt; ";
}
protected AllBranchPrintWalker forkPath() {
return new AllBranchPrintWalker();
}
}
protected BranchWalker joinBranches(List&lt;BranchWalker&gt; branchWalkers) {
// TODO Auto-generated method stub
return null;
}
protected BranchWalkerInternal firstBranchWalker() {
return new AllBranchPrintWalker();
}
}
List&lt;String&gt; getBranchStrings() {
List&lt;String&gt; branchStrings = new LinkedList&lt;&gt;();
for (GraphExplorerInternal app : getActivatedExplorers()) {
for (BranchWalkerInternal ap : app.getAllBranches()) {
AllBranchPrintWalker printPath = (AllBranchPrintWalker) ap;
branchStrings.add(printPath.curString);
}
}
return branchStrings;
}
}</code></pre>
</div>
</div>
</div>
<div class="sect4">
<h5 id="_path-quantor"><a class="anchor" href="#_path-quantor"></a><a class="link" href="#_path-quantor">22.1.3.6. Path quantor</a></h5>
<div class="paragraph">
<p>Graph explorers are typically used to reason on all branch walkers that start at a specific location.
For instance, such a reasoning might determine whether some source element is reachable or whether a variable is used or not.
To simplify this, quantors are provided.
Since branch walkers originating from a single activation point can fork, the reasoning has to include all these forked branch walkers.
Hence, graph explorers are instantiated using a quantor which can be either <em>For All</em>, <em>At Least One</em> OR <em>None</em> that refers to all branches.
After all branch walkers of an explorer are terminated, the explorer is regarded as either passed or failed.
Paths also can be aborted manually using the methods <code>pass()</code> or <code>fail()</code>.
When <em>pass</em> or <em>fail</em> are used, the graph explorer might be terminated in the following cases:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>If the quantor of the graph explorer is <em>For All</em>, and <code>fail()</code> is called on a branch walker.</p>
</li>
<li>
<p>If the quantor of the graph explorer is <em>At Least One</em>, and <code>pass()</code> is called on a branch walker.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Additionally, a graph explorer can be aborted manually by canceling all its branches.</p>
</div>
</div>
</div>
<div class="sect3">
<h4 id="_control-flow-analyses"><a class="anchor" href="#_control-flow-analyses"></a><a class="link" href="#_control-flow-analyses">22.1.4. Control flow analyses</a></h4>
<div class="sect4">
<h5 id="_dead-code-analysis"><a class="anchor" href="#_dead-code-analysis"></a><a class="link" href="#_dead-code-analysis">22.1.4.1. Dead code analysis</a></h5>
<div class="paragraph">
<p>The dead code analysis uses the graph visitor in all four modes and collects all visited CF elements.
The collected CF elements are saved separately for every mode.
After the graph visitor is terminated, the unreachable CF elements are computed like follows:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>CF elements, that are collected during forward and catch block mode are reachable.</p>
</li>
<li>
<p>CF elements, that are collected during islands mode are unreachable.</p>
</li>
<li>
<p>CF elements, that are <em>only</em> collected during backward mode, are also unreachable.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>In a later step, the unreachable elements are merged into unreachable text regions that are used for error markers.</p>
</div>
</div>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:dataflow"><a class="anchor" href="#sec:dataflow"></a><a class="link" href="#sec:dataflow">22.2. Dataflow</a></h3>
<div class="paragraph">
<p></p>
</div>
<div class="sidebarblock">
<div class="content">
<a href="https://github.com/eclipse/n4js/issues/331" class="gray" title="GitHub Issues"><span class="image"><img src="images/issue.svg" alt=""></span> #331</a>
<a href="https://github.com/eclipse/n4js/issues/464" class="gray" title="GitHub Issues"><span class="image"><img src="images/issue.svg" alt=""></span> #464</a>
</div>
</div>
<div class="sect3">
<h4 id="_dataflow-graph"><a class="anchor" href="#_dataflow-graph"></a><a class="link" href="#_dataflow-graph">22.2.1. Dataflow graph</a></h4>
<div class="paragraph">
<p>The data flow graph provides means to reason about <em>symbols</em>, <em>effects</em>, <em>data flow</em>, <em>aliases</em> and <em>guards</em> in the control flow graph.
The main classes of the data flow API are <code>DataflowVisitor</code> and <code>Assumption</code>.</p>
</div>
<div class="dlist">
<dl>
<dt class="hdlist1">Symbol</dt>
<dd>
<p>Symbols represent a program variable in the sence that it represents all AST elements, that bind to the same variable declaration (according to scoping).
The terms <em>symbol</em> and <em>variable</em> are used synonymously.</p>
</dd>
<dt class="hdlist1">Effect</dt>
<dd>
<p>Effects are reads, writes and declarations of symbols.
For instance, a typical CF element with a write effect is an assignment such as <code>a = null;</code>.
Every effect refers to a single symbol and graph node.
The following effects are provided:</p>
<div class="ulist">
<ul>
<li>
<p><em>Declaration</em>: is the declaration of a variable.</p>
</li>
<li>
<p><em>Write</em>: is the definition of a variable&#8217;s value, which is typically done with an assignment.</p>
</li>
<li>
<p><em>Read</em>: is the read of a variable&#8217;s value, which could happen when passing a variable as an argument to a method call.</p>
</li>
<li>
<p><em>MethodCall</em>: is the call of a property method of a variable.</p>
</li>
</ul>
</div>
</dd>
</dl>
</div>
<div class="paragraph">
<p>Note that the term <em>value use</em> means either write or method call of a variable.
The term <em>value definition</em> means that a variable is written.</p>
</div>
<div class="dlist">
<dl>
<dt class="hdlist1">Data flow</dt>
<dd>
<p>The term data flow is used for assignments of all kind.
For instance, the assigments <code>a = b</code>, <code>a = 1</code>, <code>a = null</code> or even <code>for (let [a] of [[0],[undefined]]);</code> are data flows.
The data is always flowing from the right hand side to the left hand side.</p>
</dd>
<dt class="hdlist1">Alias</dt>
<dd>
<p>Due to data flow, other symbols can get important for an analysis.
For instance, the data flow <code>a = b</code> makes <code>b</code> important when reasoning about <code>a</code> since the value of <code>b</code> is assigned to <code>a</code>.
In the API is <code>b</code> therefore called an alias of <code>a</code>.</p>
</dd>
<dt class="hdlist1">Guard</dt>
<dd>
<p>Guards are conditions that appear in e.g. <code>it</code>-statements.
For instance, a typical guard is the null-check in the following statement: <code>if (a == null) foo();</code>.
For every CF element, guards can hold either <em>always</em>, <em>never</em> or <em>sometimes</em>.
Note that the null-check-guard always holds at the method invocation <code>foo();</code>.</p>
</dd>
<dt class="hdlist1"><code>DataflowVisitor</code></dt>
<dd>
<p>The class <code>DataflowVisitor</code> provides means to visit all code locations where either effects happen or guards are declared.
For instance, when a variable is written, the callback method <code>DataflowVisitor#visitEffect(EffectInfo effect, ControlFlowElement cfe)</code> gets called.
In case a guard is declared, the callback method <code>visitGuard(Guard guard)</code> gets called.</p>
</dd>
<dt class="hdlist1"><code>Assumption</code></dt>
<dd>
<p>The class <code>Assumption</code> provides means to track the data flow of a specific symbol from a specific code location.
For instance, assumptions are used to detect whether the symbol <code>s</code> in the property access <code>s.prop</code> is or may be undefined.
In this example, the assumption symbol is <code>s</code> and its start location is the property access.
From there, the data flow of <code>s</code> is tracked in backwards traverse direction.
Also, (transitive) aliases of <code>s</code> are tracked.
In case a data flow that happens on <code>s</code> or its aliases, the callback method <code>holdsOnDataflow(Symbol lhs, Symbol rSymbol, Expression rValue)</code> is called.
For every effect that affects <code>s</code> or one of its aliases, the callback method <code>holdsOnEffect(EffectInfo effect, ControlFlowElement container)</code> is called.
And finally, for all guards that hold always/never at the start location regarding symbol <code>s</code>, the callback method <code>holdsOnGuards(Multimap&lt;GuardType, Guard&gt; neverHolding, Multimap&lt;GuardType, Guard&gt; alwaysHolding)</code> is called.</p>
</dd>
</dl>
</div>
</div>
<div class="sect3">
<h4 id="_dataflow-analyses"><a class="anchor" href="#_dataflow-analyses"></a><a class="link" href="#_dataflow-analyses">22.2.2. Dataflow analyses</a></h4>
<div class="sect4">
<h5 id="_def-def-def-nothing-analysis"><a class="anchor" href="#_def-def-def-nothing-analysis"></a><a class="link" href="#_def-def-def-nothing-analysis">22.2.2.1. Def&#8594;Def / Def&#8594;Nothing analysis</a></h5>
<div class="paragraph">
<p>A Def&#8594;Def analysis finds all defintions of a variable that are always a predecessor of another definition.
Its result is a set of all obsolete definition sites.</p>
</div>
<div class="paragraph">
<p>A Def&#8594;!Use analysis finds all definitions of a variable that are not followed by either a read or a method call.
These definition are therefore obsolete and can be removed.</p>
</div>
<div class="paragraph">
<p>Both of these analyses are performed in traverse direction <em>Forward</em>.</p>
</div>
</div>
<div class="sect4">
<h5 id="_def-use-decl-analysis"><a class="anchor" href="#_def-use-decl-analysis"></a><a class="link" href="#_def-use-decl-analysis">22.2.2.2. Def|Use&#8592;Decl analysis</a></h5>
<div class="paragraph">
<p>A Def|Use&#8592;Decl analysis finds all preceding <em>def</em> or <em>use</em> sites of a declarations of a specific variable.
The paths might contain other <em>defs</em> or <em>uses</em> of the same variable.
In case such paths exists, the variable is used before it is declared.
This analysis is done in traverse direction <em>Backward</em>.</p>
</div>
<div id="img:usedBeforeDeclaredAnalysis" class="imageblock">
<div class="content">
<img src="chapters/30_flowgraphs/images/usedBeforeDeclaredAnalysis.png" alt="usedBeforeDeclaredAnalysis">
</div>
<div class="title">Figure 60. Finding use or def sites can be done using the graph visitor in traverse direction <em>Backward</em>.</div>
</div>
<div class="paragraph">
<p>In the <a href="#img:usedBeforeDeclaredAnalysis">above figure</a> a graph visitor would visit all CF elements.
When it visits the declaration in line 8 (<code>let w</code>), it will activate a graph explorer (star 1 in the figure) for variable <code>w</code>.
Now, the first branch walker <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>b</mi><mn>1</mn></msub></math> is created and walks the control in backward traverse direction.
When <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>b</mi><mn>1</mn></msub></math> encounters the exit node of the <code>if</code>-statement, it will create two forked branches <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>b</mi><mn>2</mn></msub></math> and <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>b</mi><mn>3</mn></msub></math>.
Now, <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>b</mi><mn>2</mn></msub></math> enters then the branch of the <code>if</code>-statement (star 2), while <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>b</mi><mn>3</mn></msub></math> traverses directly to the condition of the <code>if</code>-statement.
Next, <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>b</mi><mn>2</mn></msub></math> visits the def site of variable <code>w</code> (star 3).
This means, that there exist a def site of <code>w</code> before <code>w</code> was declared and hence, an error should be shown.
Since there could exist more cases like this, neither the branch walker nor the graph explorer are terminated.
When reaching star 4, the two branch walkers <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>b</mi><mn>2</mn></msub></math> and <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>b</mi><mn>3</mn></msub></math> are joined and the follow-up branch walker <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>b</mi><mn>4</mn></msub></math> is created.
At star 5, the end the CF container is reached and the <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>b</mi><mn>4</mn></msub></math> will be terminated.
After all branch walkers are terminated, the graph explorer for the declaration site of variable <code>w</code> is evaluated:
All use or def sites, that were reachable should be marked with an error saying that the declaration has to be located before the use of a variable.</p>
</div>
<div class="paragraph">
<p>Note this analysis is currently implemented as a control flow analysis since it does not rely on guards, aliases.
Also, it only relies on local variables and hence does not need the symbols that are provided by the data flow API.</p>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="sec:publish-npms-to-public"><a class="anchor" href="#sec:publish-npms-to-public"></a><a class="link" href="#sec:publish-npms-to-public">23. Publish npms</a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>We publish npms located in the folder <code>n4js-libs</code> to the <a href="registry.npmjs.org">public npm registry</a>. Specifically, the following npms are published:</p>
</div>
<div class="dlist">
<dl>
<dt class="hdlist1">Command line tools</dt>
<dd>
<div class="ulist">
<ul>
<li>
<p>n4js-cli</p>
</li>
<li>
<p>n4js-mangelhaft-cli</p>
</li>
</ul>
</div>
</dd>
<dt class="hdlist1">Runtime definition files</dt>
<dd>
<div class="ulist">
<ul>
<li>
<p>n4js-runtime-ecma402</p>
</li>
<li>
<p>n4js-runtime-es2015</p>
</li>
<li>
<p>n4js-runtime-esnext</p>
</li>
<li>
<p>n4js-runtime-fetch</p>
</li>
<li>
<p>n4js-runtime-html5</p>
</li>
<li>
<p>n4js-runtime-node</p>
</li>
<li>
<p>n4js-runtime-v8</p>
</li>
</ul>
</div>
</dd>
<dt class="hdlist1">Mangelhaft</dt>
<dd>
<div class="ulist">
<ul>
<li>
<p>org.eclipse.n4js.mangelhaft</p>
</li>
<li>
<p>org.eclipse.n4js.mangelhaft.assert</p>
</li>
<li>
<p>org.eclipse.n4js.mangelhaft.reporter.console</p>
</li>
<li>
<p>org.eclipse.n4js.mangelhaft.reporter.ide</p>
</li>
<li>
<p>org.eclipse.n4js.mangelhaft.reporter.xunit</p>
</li>
<li>
<p>org.eclipse.n4js.mangelhaft.runner.ide</p>
</li>
</ul>
</div>
</dd>
</dl>
</div>
<div class="paragraph">
<p>In order to make sure that the npms work correctly with the <code>n4js</code> product, we need to integration test the interplay between the n4js products and the npms. Right now, we only focus on the interplay between the <code>n4js</code> headless compiler and npms. For integration tests, we publish the npms to a local npm registry which is provided by <a href="https://www.verdaccio.org/docs/en/docker.html">verdaccio docker image</a> before executing the tests. When all integration tests are executed, we stop the local npm registry.</p>
</div>
<div class="sect2">
<h3 id="sec:publish-npms-n4js-maven"><a class="anchor" href="#sec:publish-npms-n4js-maven"></a><a class="link" href="#sec:publish-npms-n4js-maven">23.1. Publish n4js-libs to during maven build</a></h3>
<div class="admonitionblock warning">
<table>
<tr>
<td class="icon">
<i class="fa icon-warning" title=""></i>
</td>
<td class="content">
The NPMs are currently published using NumberFour&#8217;s internal build infrastructure in combination with extended integration tests. This needs to be changed in the future!
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>This section describes how integration tests can use local npm registry during the test.
All integration tests that require a local npm registry should be placed in the bundle <code>org.eclipse.n4js.hlc.integrationtests</code></p>
</div>
<div class="imageblock center">
<div class="content">
<img src="chapters/40_publish_npms/images/publish_npm_in_mvn_workflow.svg" alt="publish npm in mvn workflow">
</div>
<div class="title">Figure 61. Maven phases</div>
</div>
<div class="paragraph">
<p>During the maven build, three projects <code>org.eclipse.n4js.external.libraries.update</code>, <code>org.eclipse.n4js.product.build</code> and <code>org.eclipse.n4js.hlc.integrations</code> are built by maven in that order.</p>
</div>
<div class="ulist">
<ul>
<li>
<p>During the phase <em>process-classes</em> of the <code>org.eclipse.n4js.external.libraries.update</code> build, the UpdateShippedCode MWE2 workflow is triggerd to compile n4js code of npms in n4js-lib. The phase <em>process-classes</em> was chosen because it must happen after the <code>org.eclipse.n4js.external.libraries.update</code> bundle has been compiled.</p>
</li>
<li>
<p>During the <em>verify</em> phase of the <code>org.eclipse.n4js.product.build</code> build, the Maven <code>exec-maven</code> plugin calls the script <code>n4js/releng/utils/scripts/publish-n4js-libs.sh</code> to publish the npms in the n4js-lib to the <em>staging npm registry</em>. The URL of this staging npm registry must be configured before triggering maven build via the environment variable <code>NPM_STAGING_REGISTRY</code>. Note that the staging npm registry lives beyond the life of <code>n4js-inhouse</code> 's maven build and holds npms that are needed by the integration tests in the n4js-extended&#8217;s build.</p>
</li>
<li>
<p>During the <em>pre-integration-test</em> phase of the <code>org.eclipse.n4js.hlc.integrationtests</code> bundle, the Maven <code>antrun</code> plugin starts a <em>verdaccio</em> docker container local npm registry at <code><a href="http://localhost:4873" class="bare">http://localhost:4873</a></code> via docker. Also in the very same phase, the Maven plugin <code>exec-maven</code> calls the script <code>n4js/releng/utils/scripts/publish-n4js-libs.sh</code> to publish the npms in the n4js-lib folder to the local registry <code><a href="http://localhost:4873" class="bare">http://localhost:4873</a></code>. The list of published npms is identical to that list above. Note that the npms are published with the <code>dist-tag</code> <em>test</em>.</p>
</li>
<li>
<p>During the <em>integration-test</em> phase of <code>org.eclipse.n4js.hlc.integrationtests</code>, the Maven <code>failsafe</code> plugin executes the integration tests. Here, the integration tests can pull the required npms from the local registry populated during the <em>pre-integration-test</em> above.</p>
</li>
<li>
<p>In the <em>pre-integration-test</em> phase of <code>org.eclipse.n4js.hlc.integrationtests</code>, the Maven <code>antrun</code> plugins removes the <em>verdaccio</em> docker container.</p>
</li>
</ul>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="sec:Hints"><a class="anchor" href="#sec:Hints"></a><a class="link" href="#sec:Hints">Appendix A: Hints</a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>In this chapter, some tips and tricks regarding Eclipse, Xtend and Maven should be collected.</p>
</div>
<div class="sect2">
<h3 id="sec:XtextInjection"><a class="anchor" href="#sec:XtextInjection"></a><a class="link" href="#sec:XtextInjection">A.1. Xtext Injection</a></h3>
<div class="paragraph">
<p><a href="#fig:cd_XtextInjectors">[fig:cd_XtextInjectors]</a> shows different injectors used by Xtext
and their relation to the injector of a custom language created with Xtext
(in this example N4JS).</p>
</div>
<div id="fig:XtextInjectors" class="imageblock center">
<div class="content">
<img src="chapters/a02_hints/images/cd_XtextInjectors.svg" alt="cd XtextInjectors">
</div>
<div class="title">Figure 62. Xtext injectors and custom DSL injector</div>
</div>
<div class="paragraph">
<p><strong>Injectors creation:</strong></p>
</div>
<div class="ulist">
<ul>
<li>
<p>create 'SharedInjector'</p>
<div class="ulist">
<ul>
<li>
<p>create shared singletons</p>
</li>
</ul>
</div>
</li>
<li>
<p>create (lazily) custom language injector</p>
<div class="ulist">
<ul>
<li>
<p>take singletons from shared injector</p>
</li>
<li>
<p>add bindings from 'SharedModule'</p>
</li>
<li>
<p>create own singletons</p>
</li>
</ul>
</div>
</li>
</ul>
</div>
<div class="paragraph">
<p>Normally one injector is bound to one language.
'ContributingModule' allows custom languages to contribute bindings to the
shared state, effectively cross project boundaries.</p>
</div>
<div class="paragraph">
<p>It must be noted that in case of N4JS tools there are multiple languages
contributing / extending Xtext injector, which can be seen in figure
<a href="#fig:cd_customInjectors">Xtext injectors and custom DSL injector</a></p>
</div>
<div id="fig:cd_customInjectors" class="imageblock center">
<div class="content">
<img src="chapters/a02_hints/images/cd_customInjectors.svg" alt="cd customInjectors">
</div>
<div class="title">Figure 63. Xtext injectors and custom DSL injector</div>
</div>
<div class="sect3">
<h4 id="sec:DI_MultipleInjectors_Singletons"><a class="anchor" href="#sec:DI_MultipleInjectors_Singletons"></a><a class="link" href="#sec:DI_MultipleInjectors_Singletons">A.1.1. Multiple Injectors and Singletons</a></h4>
<div class="paragraph">
<p>Every injector creates its 'ObjectGraph'. Having multiple Injectors in
the system leads to multiple (disconnected) object graphs. For normal instances
that is not an issue, but for scoped instances this causes problems.
Most common issue happens with '@Singleton' instances that carry state.</p>
</div>
<div id="fig:cd_SingletonDuplicate" class="imageblock center">
<div class="content">
<img src="chapters/a02_hints/images/cd_SingletonDuplicate.svg" alt="cd SingletonDuplicate">
</div>
<div class="title">Figure 64. Xtext injectors and custom DSL injector</div>
</div>
<div class="paragraph">
<p><a href="#fig:cd_customInjectors">Xtext injectors and custom DSL injector</a> shows situation in which both 'ChildInjector'
and 'N4JSInjector' have their bindings for 'N4JSEclipseCore'. As a result those
injectors will create their instance of core that is expected to be
'@Singleton'. Additionally, this will be true for all its transitive
dependencies.</p>
</div>
<div class="sect4">
<h5 id="sec:DI_avoid_duplicate_singletons"><a class="anchor" href="#sec:DI_avoid_duplicate_singletons"></a><a class="link" href="#sec:DI_avoid_duplicate_singletons">A.1.1.1. Avoiding duplicate singletons</a></h5>
<div class="paragraph">
<p>To avoid issue with duplicate singletons two distinct injectors should not
have their bindings for singletons. Developer needs to decide where to
define <strong>the only</strong> binding, and let one 'ObjectGraph' delegate to another.</p>
</div>
<div class="sect5">
<h6 id="sec:DI_binding_in_shared"><a class="anchor" href="#sec:DI_binding_in_shared"></a><a class="link" href="#sec:DI_binding_in_shared">A.1.1.1.1. Defining binding in the shared injector</a></h6>
<div class="paragraph">
<p>One approach is to define binding in the shared injector. Then in the injector
of the custom language to delegate to the shared contribution.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>/** Binds {@link IN4JSCore} */
public class ContributingModule implements Module {
@Override
public void configure(Binder binder) {
binder.bind(IN4JSCore.class).to(IN4JSEclipseCore.class);
binder.bind(IN4JSEclipseCore.class).to(N4JSEclipseCore.class).in(SINGLETON);
}
}
/** Delegates binding for {@link IN4JSCore} to the shared provider. */
public class ContributingModule implements Module {
public Provider&lt;? extends IN4JSCore&gt; bindIN4JSCore() {
return Access.contributedProvider(N4JSEclipseCore.class);
}
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>Downside of this approach is in the shared injector itself.
It does not allow for implicit bindings. This forces developer to declare
bindings for <strong>all transitive</strong> dependencies of the main binding explicitly.
Additionally, every custom language has to do it. These make shared injector
the <em>GodInjector</em> that contains configuration for all custom languages,
it is responsible for creating most objects in the system, and potentially
exposes types from one language to another language where it might not be
desired.</p>
</div>
</div>
<div class="sect5">
<h6 id="sec:DI_binding_in_custom"><a class="anchor" href="#sec:DI_binding_in_custom"></a><a class="link" href="#sec:DI_binding_in_custom">A.1.1.1.2. Defining binding in the custom injector</a></h6>
<div class="paragraph">
<p>Other approach is to define binding in the injector for a custom
language. Then let instances in the shared injector object graph to obtain
singleton instances via custom language injector (which is stored on the
custom language activator).</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>/** Does not bind {@link IN4JSCore} */
public class ContributingModule implements Module {
@Override
public void configure(Binder binder) {
// no core binding
}
}
/** Binds {@link IN4JSCore}. */
public class ContributingModule implements Module {
public Class&lt;? extends IN4JSCore&gt; bindIN4JSCore() {
return IN4JSEclipseCore.class;
}
}
/** Some type used in shared injector object graph */
public SomeSharedType{
/** Obtain {@link IN4JSCore} form {@code N4JSInjector}. */
private IN4JSCore getIN4JSCore() {
return N4JSActivator
.getInstance()
.getInjector(ORG_ECLIPSE_N4JS_N4JS)
.getInstance(IN4JSCore.class);
}
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>This approach also has downsides. In the 'SomeSharedType' that exists in the
shared injector object graph we cannot inject 'IN4JSCore' as it is not
known to the shared injector. Instead, we have to get the instance form the
'N4JSInjector' manually. This requires developer to know whole (singleton)
types structure
defined in every custom language.</p>
</div>
</div>
</div>
</div>
<div class="sect3 language-n4js">
<h4 id="sec:DI_Hints"><a class="anchor" href="#sec:DI_Hints"></a><a class="link" href="#sec:DI_Hints">A.1.2. Dependency Injection Hints</a></h4>
<div class="sect4">
<h5 id="sec:DI_custom_bundle"><a class="anchor" href="#sec:DI_custom_bundle"></a><a class="link" href="#sec:DI_custom_bundle">A.1.2.1. Use DI in custom bundle, use DI with extensions</a></h5>
<div class="sect5">
<h6 id="sec:DI_custom_bundle_problem"><a class="anchor" href="#sec:DI_custom_bundle_problem"></a><a class="link" href="#sec:DI_custom_bundle_problem">A.1.2.1.1. Problem</a></h6>
<div class="paragraph">
<p>DI should be used in a custom bundle, i.e. a bundle not generated by Xtext.
E.g., a new handler should be provided in its plugin, and this handler requires
an injected instance. Example</p>
</div>
<div class="paragraph">
<p>my.dsl.bundle.ui xtext generated</p>
</div>
<div class="paragraph">
<p>my.dsl.bundle.sub.ui
The following class is contained in my custom plugin:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>class my.dsl.bundle.sub.ui.Handler {
@Inject SomeDSLOrXtextSpecificType obj;
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>The question is, how can obj of type be injected at this location?</p>
</div>
</div>
<div class="sect5">
<h6 id="sec:DI_custom_bundle_solution"><a class="anchor" href="#sec:DI_custom_bundle_solution"></a><a class="link" href="#sec:DI_custom_bundle_solution">A.1.2.1.2. Solution</a></h6>
<div class="paragraph">
<p>First of all, to use DI in a type, the type instance itself must have been
created via DI. This requires an injector which uses the same class loader as
the type using the injector. This means that a new bundle needs its injector,
created by an IExecutableExtensionFactory using the bundles' activator (plugin)
singleton.</p>
</div>
<div class="paragraph">
<p>This activator can extend the generated activator of a Xtext bundle. The
following code can be used as a template, as long as no custom non-default
bindings are to be added (in this case, have a look at the generated activator
and override the methods configuring the injector):</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>public class my.dsl.bundle.sub.ui.Activator extends my.dsl.bundle.ui.MyDSLActivator {
private static my.dsl.bundle.sub.ui.Activator INSTANCE;
@Override
public void start(BundleContext context) throws Exception {
super.start(context);
INSTANCE = this;
}
@Override
public void stop(BundleContext context) throws Exception {
INSTANCE = null;
super.stop(context);
}
public static TypePopupActivator getInstance() {
return INSTANCE;
}
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>Additionally, a custom 'AbstractGuiceAwareExecutableExtensionFactory' has to be
implemented. This class then uses the new activator instance (this is required
as bundles have their classloaders!)</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>public class my.dsl.bundle.sub.ui.SubExecutableExtensionFactory extends AbstractGuiceAwareExecutableExtensionFactory {
@Override
protected Bundle getBundle() {
return my.dsl.bundle.sub.ui.Activator.getInstance().getBundle();
}
@Override
protected Injector getInjector() {
return my.dsl.bundle.sub.ui.Activator.getInstance().getInjector(MyDSLActivator.MY_LANGUAGE_GRAMMAR);
}
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>Now, we can use this extension factory in the plugin.xml of the sub bundle to
let the handler be created via DI. E.g.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>"org.eclipse.ui.handlers"&amp;gt;
&lt;handler
class="my.dsl.bundle.sub.ui.SubExecutableExtensionFactory:my.dsl.bundle.sub.ui.Handler"
commandId="..."&amp;gt;
handler&amp;gt;
extension&amp;gt;</code></pre>
</div>
</div>
</div>
</div>
<div class="sect4">
<h5 id="sec:Access_Other_DSL_Injector"><a class="anchor" href="#sec:Access_Other_DSL_Injector"></a><a class="link" href="#sec:Access_Other_DSL_Injector">A.1.2.2. How do I get the Guice Injector of my language?</a></h5>
<div class="paragraph">
<p>We have the use case to load a N4MF file inside the N4JS infrastructure to read
out the project description and configure the qualified names and container
visibility. I.e. we have to load another DSL in our current DSL infrastructure,
in the use case to have a Xtext resource set available to load the N4MF file.
Injecting the Xtext resource of the current DSL wouldn’t work as it has not the
N4MF injection context. So in the following the ways how to access this
injection context is described as extracted from
<a href="http://koehnlein.blogspot.de/2012/11/xtext-tip-how-do-i-get-guice-injector.html">this blog post</a>.</p>
</div>
<div class="sect5">
<h6 id="sec:DSL_Injector_UI_context"><a class="anchor" href="#sec:DSL_Injector_UI_context"></a><a class="link" href="#sec:DSL_Injector_UI_context">A.1.2.2.1. UI context</a></h6>
<div class="paragraph">
<p>To access another DSL injector in a UI DSL project just add a dependency to the
UI project of the other DSL and then</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>MyClass myClass =
TheOtherDSLActivator.getInstance().getInjector().get(MyClass.class)</code></pre>
</div>
</div>
</div>
<div class="sect5">
<h6 id="sec:DSL_Injector_Non_UI_context"><a class="anchor" href="#sec:DSL_Injector_Non_UI_context"></a><a class="link" href="#sec:DSL_Injector_Non_UI_context">A.1.2.2.2. Non UI context but with injection context</a></h6>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>@Inject IResourceServiceProvider.Registry serviceProviderRegistry;
...
MyClass myClass
=
serviceProviderRegistry.getResourceServiceProvider(URI.createFileURI(n4mfFileAbsolutePath)).get(MyClass.class)</code></pre>
</div>
</div>
</div>
<div class="sect5">
<h6 id="sec:DSL_Injector_Non_UI_non_injection_context"><a class="anchor" href="#sec:DSL_Injector_Non_UI_non_injection_context"></a><a class="link" href="#sec:DSL_Injector_Non_UI_non_injection_context">A.1.2.2.3. Non UI context without injection context</a></h6>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>@Inject IResourceServiceProvider.Registry serviceProviderRegistry;
...
MyClass
myClass
=
IResourceServiceProvider.Registry.INSTANCE.getResourceServiceProvider(uri).get(MyClass.class);</code></pre>
</div>
</div>
</div>
</div>
<div class="sect4">
<h5 id="sec:Cancel_Indicator"><a class="anchor" href="#sec:Cancel_Indicator"></a><a class="link" href="#sec:Cancel_Indicator">A.1.2.3. How do I get cancel indicators in different contexts?</a></h5>
<div class="paragraph">
<p>Several factors contribute to responsiveness in the IDE, but here we focus in
running jobs in the background and reacting to cancellation requests.</p>
</div>
<div class="paragraph">
<p>The Eclipse Jobs API is recommended for potentially long-running tasks (other
than incremental building, which has dedicated support). For example, the
outline view is populated by a background job, running validations on the
resource (and honoring cancellation requests initiated as for any job).</p>
</div>
<div class="paragraph">
<p>Cancel indicators are a Xtext abstraction while Eclipse favors progress
monitors, the latter including not only cancellation capability but also a
callback mechanism to give feedback in the UI about intermediate progress.
Cancel indicator can wrap a progress monitor.</p>
</div>
<div class="paragraph">
<p>Cancel indicators come in two variants, depending on the source of cancellation
events:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>a resource becoming stale (usually as a result of editing sources) triggers
cancellation. These cancel indicators can be obtained via
'OutdatedStateManager', which itself is available via injection.</p>
</li>
<li>
<p>cancel indicators associated to the UI, for example associated to an Eclipse
job. Examples:</p>
<div class="ulist">
<ul>
<li>
<p>for an outline view running in the background, an override of method
'createRoot()' from 'DefaultOutlineTreeProvider' receives a UI-aware cancel
indicator;</p>
</li>
<li>
<p>for the transpiler, instances that carry cancel indicator are
'IFileSystemAccess' and (in the future) 'IGenerator2'. To track the latter, see
Eclipse bug 477068.</p>
</li>
</ul>
</div>
</li>
</ul>
</div>
<div class="paragraph">
<p>In general, whenever a resource is validated cancel indicator should be checked
periodically. These checks are performed automatically via
'MethodWrapperCancelable' before the (reflective) invocation of each validation
method and therefore require no manual intervention, see
'AbstractMessageAdjustingN4JSValidator'. However, that doesn’t help in case a
single validation method ''takes too long''. To simplify those checks, utility
'isCanceled()' of 'AbstractMessageAdjustingN4JSValidator' can be invoked.</p>
</div>
</div>
</div>
</div>
<div class="sect2">
<h3 id="sec:Eclipse"><a class="anchor" href="#sec:Eclipse"></a><a class="link" href="#sec:Eclipse">A.2. Eclipse</a></h3>
<div class="sect3">
<h4 id="sec:Show_Xtext_Index"><a class="anchor" href="#sec:Show_Xtext_Index"></a><a class="link" href="#sec:Show_Xtext_Index">A.2.1. Show the current Xtext index</a></h4>
<div class="paragraph">
<p>Press the following keyboard shortcut in the running UI: <kbd>CTRL</kbd><br>
<kbd>SHIFT</kbd> + <kbd>F3</kbd> (likely under Mac <kbd>CMD</kbd> + <kbd>SHIFT</kbd> + <kbd>F3</kbd>).</p>
</div>
</div>
<div class="sect3">
<h4 id="sec:Plugin_spy"><a class="anchor" href="#sec:Plugin_spy"></a><a class="link" href="#sec:Plugin_spy">A.2.2. Plug-in spy</a></h4>
<div class="paragraph">
<p>Not special for Xtext but very helpful do identify which class implements a UI
concept, for example, if you want to know which class implements the Open Model
Element dialog just press <kbd>CTRL</kbd> + <kbd>SHIFT</kbd> + <kbd>F3</kbd> to open that
dialog and afterwards press <kbd>SHIFT</kbd> + <kbd>ALT</kbd> + <kbd>F1</kbd> to show that
'XtextEObjectSearchDialog' is used as implementation. Additionally, use
<kbd>SHIFT</kbd> + <kbd>ALT</kbd> + <kbd>F2</kbd> to spy buttons in the toolbar and
<kbd>SHIFT</kbd> + <kbd>ALT</kbd> + <kbd>F3</kbd> to spy the extension point name of the
currently active view or window.</p>
</div>
</div>
</div>
<div class="sect2">
<h3 id="sec:Maven-hints"><a class="anchor" href="#sec:Maven-hints"></a><a class="link" href="#sec:Maven-hints">A.3. Maven</a></h3>
<div class="sect3">
<h4 id="how-to-check-for-maven-mojo-updates"><a class="anchor" href="#how-to-check-for-maven-mojo-updates"></a><a class="link" href="#how-to-check-for-maven-mojo-updates">A.3.1. How to check for Maven MOJO updates</a></h4>
<div class="paragraph">
<p><strong>cd</strong> to the root directory and call</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-bash" data-lang="bash">mvn versions:display-plugin-updates</code></pre>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_module-loading"><a class="anchor" href="#_module-loading"></a><a class="link" href="#_module-loading">Appendix B: Module Loading</a></h2>
<div class="sectionbody">
<div class="admonitionblock warning">
<table>
<tr>
<td class="icon">
<i class="fa icon-warning" title=""></i>
</td>
<td class="content">
This chapter is outdated and basically kept for historical reasons.
</td>
</tr>
</table>
</div>
<div class="sect2">
<h3 id="sec:Dependency_Management"><a class="anchor" href="#sec:Dependency_Management"></a><a class="link" href="#sec:Dependency_Management">B.1. Dependency Management</a></h3>
<div class="paragraph">
<p>There exist several types of dependencies between modules, distinguishable by the time when the dependency is relevant. We first define these dependencies lazily to give an impression of the problem, at more rigorously later on.</p>
</div>
<div class="paragraph">
<p>Dependency needed at compile time. These type of dependency is removed by the compiler. These are basically type references used in variable or function declarations.</p>
</div>
<div class="paragraph">
<p>Runtime dependencies are to be handled at runtime in general. We distinguish two special types of runtime dependencies:</p>
</div>
<div class="paragraph">
<p>A loadtime dependency is a special runtime dependency that needs to be resolved before a module is initialized, that is, when all top-level statements of a module, containing class declarations, are executed. This usually is a types super type (e.g., super class), or a call to a function (defined in a different module) in a static initializer or module top level statement.</p>
</div>
<div class="paragraph">
<p>An execution time dependency is a non-initialization runtime dependency. That is, when a method is called (from another module), this is execution time.</p>
</div>
<div class="paragraph">
<p>Of course, before a module can be loaded, it needs to be fetched (i.e., the actual code has to be retrieved by the browser).</p>
</div>
<div class="paragraph">
<p>We can define sets containing modules which a given module depends on. Note that these sets contain each other, as shown in <a href="#fig:euler_dependencies">Euler Dependencies</a>. However, we define disjoint sets in which a dependency to another type is only contained in one of the sets.</p>
</div>
<div id="fig:euler_dependencies" class="imageblock center">
<div class="content">
<img src="chapters/a10_moduleLoading/images/euler_dependencies.svg" alt="euler dependencies">
</div>
<div class="title">Figure 65. Euler diagram of dependency sets</div>
</div>
<div class="paragraph">
<p>Given a code sequence , we define the set of accessed modules in it as .</p>
</div>
<div class="literalblock">
<div class="content">
<pre>describes all function calls happening in code block , i.e. . In case calls on functions , we define a function’s body code sequence as .</pre>
</div>
</div>
<div class="paragraph">
<p>The complete set of accessed modules for a particular code sequence is then defined as
]</p>
</div>
<div class="paragraph">
<p>We explicitly allow a function to be excluded from being incorporated in the above algorithm by annotating it.</p>
</div>
<div class="paragraph">
<p>The set of load-time-dependencies for a module with initializer code is then defined as math:[\[\begin{aligned}
load-time-deps := AccessdModules( SuperClass(M) ) + AccessdModules( IC(M) ) \end{aligned}\]]</p>
</div>
</div>
<div class="sect2 language-javascript">
<h3 id="ecmascript-modules"><a class="anchor" href="#ecmascript-modules"></a><a class="link" href="#ecmascript-modules">B.2. ECMAScript Modules</a></h3>
<div class="sect3">
<h4 id="sec:ES5_Modules_Systems"><a class="anchor" href="#sec:ES5_Modules_Systems"></a><a class="link" href="#sec:ES5_Modules_Systems">B.2.1. ES5 Modules Systems</a></h4>
<div class="paragraph">
<p>Before ES6, Javascript had no built in support for modules. To overcome this hurdle, the two widely accepted formats have been :</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p><code>CommonJS</code> : Primarily aimed at synchronous module loading. The main implementation of this format is seen in <code>Node.js</code></p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-javascript" data-lang="javascript">var value = 100;
function inc() {
value++;
}
module.exports = {
value : value,
inc : inc
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>Import via require.</p>
</div>
</li>
<li>
<p><code>AMD</code> : Primarily aimed at asynchronous module loading in browsers. The main implementation of this format is seen in <code>RequireJS</code>.</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>define('myModule', ['mod1', 'mod2'], function (mod1, mod2) {
return {
myFunc: function(x, y) {
..
}
};
};
});</code></pre>
</div>
</div>
<div class="paragraph">
<p><code>passive</code> format</p>
</div>
</li>
</ol>
</div>
</div>
<div class="sect3">
<h4 id="sec:ES6_Modules"><a class="anchor" href="#sec:ES6_Modules"></a><a class="link" href="#sec:ES6_Modules">B.2.2. ES6 Modules</a></h4>
<div class="paragraph">
<p>The ES6 spec introduces <a href="http://www.ecma-international.org/ecma-262/6.0/#sec-modules">modules</a>. ES6 modules resemble <code>CommonJS</code> syntax with <code>AMD</code> like asynchronous loading support.</p>
</div>
<div class="paragraph">
<p>Apart from the syntactic details, the highlights of ES6 modules are :</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>ES6 modules support (multiple) named exports.</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>export const VERSION = "1.0.1";
export function inc(x) {
return x + 1;
}</code></pre>
</div>
</div>
</li>
<li>
<p>ES6 modules support default exports.</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>export default function (x) {
return x+1;
}</code></pre>
</div>
</div>
</li>
<li>
<p>As specified <a href="http://www.ecma-international.org/ecma-262/6.0/#sec-createimportbinding">here</a>, ES6 modules export live immutable bindings (instead of values).</p>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title=""></i>
</td>
<td class="content">
This behaviour is different from that of <code>CommonJS</code> and <code>AMD</code> modules where a snapshot of the value is exported.
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>An example demonstrating the behavioural difference :</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>//-------------src.js------------
var value = 100;
function inc() {
value++;
}
module.exports = {
value : value,
inc : inc
}
//-------------main.js------------
var src = require("./src"); //import src
console.log(src.value); //prints 100
src.inc();
console.log(src.value); //prints 100 &lt;--- The value does not update.
src.value = 65;
console.log(src.value); //prints 65 &lt;--- The imported value is mutable.</code></pre>
</div>
</div>
<div class="paragraph">
<p>The same example with ES6 modules :</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>//-------------src.js------------
export var value = 100; // &lt;--- ES6 syntax
export function inc() { // &lt;--- ES6 syntax
value++;
}
//-------------main.js------------
import {value, inc} from "src" // &lt;--- ES6 syntax
console.log(value); //prints 100
inc();
console.log(value); //prints 101 &lt;--- The value is a live binding.
value = 65; // &lt;--- throws an Error implying the binding is immutable.</code></pre>
</div>
</div>
</li>
<li>
<p>ES6 modules impose a static module structure i.e. the imports and exports can be determined at compile time (statically).</p>
</li>
</ol>
</div>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:ECMAScript_Module_Loaders"><a class="anchor" href="#sec:ECMAScript_Module_Loaders"></a><a class="link" href="#sec:ECMAScript_Module_Loaders">B.3. ECMAScript Module Loaders</a></h3>
<div class="paragraph">
<p>For resolving module dependencies and loading modules, the JS landscape provides a few different module loaders.</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p><code>RequireJS</code> is the loader of choice for in browser, <code>AMD</code> style modules. We currently transpile our code into an AMD-style format to allow it running in both Browser and Node.js environments.</p>
</li>
<li>
<p><code>Node.js</code> provides a native loader implementation for <code>CommonJS</code> style modules.</p>
</li>
<li>
<p>For browsers (primarily), tools like <a href="http://webpack.github.io/"><code>Webpack</code></a> and <a href="http://browserify.org/"><code>Browserify</code></a> exist. These tools analyse the dependency graph of the entire project and then bundle up all the dependencies in a single file. <code>Browserify</code> only supports <code>CommonJS</code> modules where as <code>Webpack</code> works with both <code>CommonJS</code> &amp; <code>AMD</code> style modules.</p>
</li>
<li>
<p>At the time of writing this document (August 2015), there does not exist any native implementation for ES6 modules by any Javascript host environments i.e. ES6 modules are not natively supported by browsers or <code>Node.js</code>, as of now.</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>[fig:moduelLoader] shows an overview.</p>
</div>
<div id="fig:moduleLoader" class="imageblock center">
<div class="content">
<img src="chapters/a10_moduleLoading/images/moduleLoader.svg" alt="moduleLoader">
</div>
<div class="title">Figure 66. Module Loader and Transpilers, Overview</div>
</div>
<div class="sect3">
<h4 id="sec:ES6_Module_Loaders"><a class="anchor" href="#sec:ES6_Module_Loaders"></a><a class="link" href="#sec:ES6_Module_Loaders">B.3.1. ES6 Module Loaders</a></h4>
<div class="paragraph">
<p>The ES6 spec started out with ES6 Module Loader details as part of the spec. However the Working Group later decided to not proceed with it. The specification for ES6 Module Loader is now a separate specification [<a href="#WhatWGLoader">WhatWGLoader</a>].</p>
</div>
<div class="paragraph">
<p>The aim of this specification is:</p>
</div>
<div class="quoteblock">
<blockquote>
<div class="paragraph">
<p>This specification describes the behavior of loading JavaScript modules from a JavaScript host environment. It also provides APIs for intercepting the module loading process and customizing loading behavior.</p>
</div>
</blockquote>
</div>
<div class="paragraph">
<p>The <a href="https://github.com/whatwg/loader#implementation-status">Implementation status</a> of the spec states :</p>
</div>
<div class="quoteblock">
<blockquote>
<div class="paragraph">
<p>It is too early to know about the Loader, first we need ES2015 modules implemented by the various engines.</p>
</div>
</blockquote>
</div>
</div>
<div class="sect3">
<h4 id="sec:Polyfills_for_ES6_Module_Loaders"><a class="anchor" href="#sec:Polyfills_for_ES6_Module_Loaders"></a><a class="link" href="#sec:Polyfills_for_ES6_Module_Loaders">B.3.2. Polyfills for ES6 Module Loaders</a></h4>
<div class="paragraph">
<p>Although there is no native support for ES6 module loading, there are a few attempts to polyfill this gap.</p>
</div>
<div class="sect4">
<h5 id="sec:es6_module_loader"><a class="anchor" href="#sec:es6_module_loader"></a><a class="link" href="#sec:es6_module_loader">B.3.2.1. es6-module-loader</a></h5>
<div class="paragraph">
<p>The <a href="https://github.com/ModuleLoader/es6-module-loader"><code>es6-module-loader</code></a> project provides a polyfill for the ES6 Module Loader implementation. It dynamically loads ES6 modules in browsers and <code>Node.js</code> with support for loading existing and custom module formats through loader hooks.</p>
</div>
</div>
<div class="sect4">
<h5 id="sec:SystemJS"><a class="anchor" href="#sec:SystemJS"></a><a class="link" href="#sec:SystemJS">B.3.2.2. SystemJS</a></h5>
<div class="paragraph">
<p>Building upon <code>es6-module-loader</code>, <a href="https://github.com/systemjs/systemjs"><code>SystemJS</code></a> supports loading ES6 modules along with <code>AMD</code>, <code>CommonJS</code> and global scripts in the browser and <code>Node.js</code>.</p>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title=""></i>
</td>
<td class="content">
In order to use ES6 modules (written in ES6 syntax) the code first needs to be transpiled to ES5. For this purpose, <code>SystemJS</code> provides an option to use <a href="https://github.com/google/traceur-compiler"><code>Traceur</code></a> compiler or <a href="https://babeljs.io/"><code>Babel</code></a>.
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="sec:Demo"><a class="anchor" href="#sec:Demo"></a><a class="link" href="#sec:Demo">B.3.2.3. Demo</a></h5>
<div class="paragraph">
<p>A demonstration of how to how to use ES6 modules with <code>Babel</code> and <code>SystemJS</code> in <code>Node.js</code> as of today (August 2015).</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Create an ES6 module as shown:</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>export var value = 100; // &lt;--- named export of a variable
export function inc() { // &lt;--- named export of a function
value++;
}</code></pre>
</div>
</div>
</li>
<li>
<p>Import the bindings from the module as shown:</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import {value, inc} from "src"
var importedValue = value; // &lt;--- using the imported value
inc(); // &lt;--- using the imported function</code></pre>
</div>
</div>
</li>
<li>
<p>Transpile these two files using <code>Babel</code> to ES5 with the target module format as <code>system</code>, as shown:</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>$ babel &lt;inputdir&gt; --out-dir &lt;outputdir&gt; --modules system</code></pre>
</div>
</div>
</li>
<li>
<p>The transpiled output should be resemble the following:</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>System.register([], function (_export) {
"use strict";
var value;
_export("inc", inc);
function inc() {
_export("value", value += 1);
}
return {
setters: [],
execute: function () {
value = 100;
_export("value", value);
}
};
});</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>System.register(["src"], function (_export) {
"use strict";
var value, inc, importedValue;
return {
setters: [function (_src) {
value = _src.value;
inc = _src.inc;
}],
execute: function () {
importedValue = value;
inc();
}
};
});</code></pre>
</div>
</div>
</li>
<li>
<p>Finally run the above transpiled files, as shown:</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>var System = require('systemjs'); // &lt;--- Require SystemJS
System.transpiler = 'babel'; // &lt;--- Configure SystemJS
System.import('main'); // &lt;--- Import the transpiled "main" module.</code></pre>
</div>
</div>
</li>
</ol>
</div>
</div>
</div>
</div>
<div class="sect2 language-typescript">
<h3 id="case-study-typescript"><a class="anchor" href="#case-study-typescript"></a><a class="link" href="#case-study-typescript">B.4. Case Study : TypeScript</a></h3>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title=""></i>
</td>
<td class="content">
<a id="sec:Case_Study___TypeScript"></a> This section is NOT an exhaustive introduction to Microsoft’s <code>TypeScript</code>, but a narrowed down analysis of certain aspects of <code>TypeScript</code>.
</td>
</tr>
</table>
</div>
<div class="sect3">
<h4 id="sec:ES6_Modules_Support"><a class="anchor" href="#sec:ES6_Modules_Support"></a><a class="link" href="#sec:ES6_Modules_Support">B.4.1. ES6 Modules Support</a></h4>
<div class="paragraph">
<p><code>TypeScript</code> language has recently added support for ES6 modules. From the <a href="https://github.com/Microsoft/TypeScript/wiki/What%27s-new-in-TypeScript#es6-modules">wiki</a> :</p>
</div>
<div class="quoteblock">
<blockquote>
<div class="paragraph">
<p><code>TypeScript</code> 1.5 supports <code>ECMAScript 6</code> (ES6) modules. ES6 modules are effectively <code>TypeScript</code> external modules with a new syntax: ES6 modules are separately loaded source files that possibly import other modules and provide a number of externally accessible exports. ES6 modules feature several new export and import declarations.</p>
</div>
</blockquote>
</div>
</div>
<div class="sect3">
<h4 id="sec:TypeScript_and_Module_Loading"><a class="anchor" href="#sec:TypeScript_and_Module_Loading"></a><a class="link" href="#sec:TypeScript_and_Module_Loading">B.4.2. TypeScript and Module Loading</a></h4>
<div class="paragraph">
<p><code>TypeScript</code> does not concern itself with providing a module loader. It is the responsibility of the host environment. However <code>TypeScript</code>’s compiler provides options to transpile the modules to different formats like <code>AMD</code>, <code>CommonJS</code>, <code>ES6</code> etc. It is the developer’s responsibility to choose an appropriate format and then use the modules with a correct module loader.</p>
</div>
<div class="paragraph">
<p>From the <a href="https://github.com/Microsoft/TypeScript/issues/2242">wiki</a> again :</p>
</div>
<div class="quoteblock">
<blockquote>
<div class="paragraph">
<p>TypeScript supports down-level compilation of external modules using the new ES6 syntax. When compiling with <code>-t ES3</code> or <code>-t ES5</code> a module format must be chosen using <code>-m CommonJS</code> or <code>-m AMD</code>. When compiling with <code>-t ES6</code> the module format is implicitly assumed to be <code>ECMAScript 6</code> and the compiler simply emits the original code with type annotations removed. When compiling down-level for <code>CommonJS</code> or <code>AMD</code>, named exports are emitted as properties on the loader supplied exports instance. This includes default exports which are emitted as assignments to exports.default.</p>
</div>
</blockquote>
</div>
<div class="paragraph">
<p>Consider the following module <code>src.ts</code> :</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>export var value = 100; //&lt;--- ES6 syntax
export function inc() { //&lt;--- ES6 syntax
value++;
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>Compiling it to <code>SystemJS</code> format produces :</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>System.register([], function(exports_1) {
var value;
function inc() {
(exports_1("value", ++value) - 1);
}
exports_1("inc", inc);
return {
setters:[],
execute: function() {
exports_1("value", value = 100); //&lt;--- ES6 syntax
}
}
});</code></pre>
</div>
</div>
<div class="paragraph">
<p>Compiling it to <code>CommonJS</code> format produces :</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>exports.value = 100; //&lt;--- ES6 syntax
function inc() {
exports.value++;
}
exports.inc = inc;</code></pre>
</div>
</div>
<div class="paragraph">
<p>Compiling it to <code>AMD</code> format produces :</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>define(["require", "exports"], function (require, exports) {
exports.value = 100; //&lt;--- ES6 syntax
function inc() {
exports.value++;
}
exports.inc = inc;
});</code></pre>
</div>
</div>
<div class="paragraph">
<p>Compiling it to <code>UMD</code> format produces :</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>(function (deps, factory) {
if (typeof module === 'object' &amp;&amp; typeof module.exports === 'object') {
var v = factory(require, exports); if (v !== undefined) module.exports = v;
}
else if (typeof define === 'function' &amp;&amp; define.amd) {
define(deps, factory);
}
})(["require", "exports"], function (require, exports) {
exports.value = 100; //&lt;--- ES6 syntax
function inc() {
exports.value++;
}
exports.inc = inc;
});</code></pre>
</div>
</div>
<div class="paragraph">
<p>NOTE :</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Visual Studio 2015 does not <a href="http://visualstudio.uservoice.com/forums/121579-visual-studio/suggestions/7017377-support-for-es6-modules">support</a> ES6 modules at this time.</p>
</li>
<li>
<p><code>SystemJS</code> supports <code>TypeScript</code> as a compiler. This implies <code>TypeScript</code> modules can be transpiled to be used with <code>SystemJS</code> as the module loader.</p>
</li>
</ol>
</div>
</div>
</div>
<div class="sect2 language-js">
<h3 id="sec:Cyclic_Dependencies"><a class="anchor" href="#sec:Cyclic_Dependencies"></a><a class="link" href="#sec:Cyclic_Dependencies">B.5. Cyclic Dependencies</a></h3>
<div class="paragraph">
<p>To better analyse and evaluate <code>SystemJS</code> module loader and different module formats, let’s look at a cyclic dependency example from a (extremely simplified) stdlib task <code>FixedPoint6</code>. The outline for the example is :</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Prepare 2 ES6 modules with a circular dependency.</p>
</li>
<li>
<p>Then transpile these modules to different module formats (e.g. <code>AMD</code>, &amp; <code>SystemJS</code>).</p>
</li>
<li>
<p>With <code>SystemJS</code> as the module loader, execute the test for every transpiled module format.</p>
</li>
</ol>
</div>
<div class="sect3">
<h4 id="sec:Setup"><a class="anchor" href="#sec:Setup"></a><a class="link" href="#sec:Setup">B.5.1. Setup</a></h4>
<div class="paragraph">
<p>Consider the following ES6 listings:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>RoundingMode</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>export default {
FLOOR : "FLOOR",
CEILING : "CEILING"
}</code></pre>
</div>
</div>
</li>
<li>
<p>MathContext</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import { default as FixedPoint6 } from "FixedPoint6";
import { default as RoundingMode } from "RoundingMode";
let MathContext = class {
constructor(mode) {
this.mode = mode;
}
divide(fp1, fp2) {
var quotient = FixedPoint6.getQuotient(fp1, fp2);
if(this.mode === RoundingMode.CEILING) {
return new FixedPoint6(Math.ceil(quotient));
} else if(this.mode === RoundingMode.FLOOR) {
return new FixedPoint6(Math.floor(quotient));
} else {
throw new Error("Incorrect RoundingMode");
}
}
}
MathContext.FLOOR = new MathContext(RoundingMode.FLOOR);
MathContext.CEILING = new MathContext(RoundingMode.CEILING);
export default MathContext;</code></pre>
</div>
</div>
</li>
<li>
<p>FixedPoint6</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import { default as MathContext } from "MathContext";
export default class FixedPoint6 {
constructor(number) {
this.value = number;
}
static getQuotient(fp1, fp2) {
return fp1.value/fp2.value;
}
divide(fp) {
return FixedPoint6.defaultContext.divide(this, fp);
}
}
FixedPoint6.defaultContext = MathContext.FLOOR;</code></pre>
</div>
</div>
</li>
<li>
<p>Test</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import { default as FixedPoint6 } from "FixedPoint6";
import { default as MathContext } from "MathContext";
import { default as RoundingMode } from 'RoundingMode';
var fp1 = new FixedPoint6(20.5);
var fp2 = new FixedPoint6(10);
var fp3 = fp1.divide(fp2);
console.log(fp1, fp2, fp3);</code></pre>
</div>
</div>
</li>
<li>
<p>Runner : This is the runner file to execute the test (after transpilation).</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>var System = require('systemjs');
System.transpiler = 'babel';
System.config({
baseURL: './build',
"paths": {
"*": "*.js"
}
});
System.import('test').catch(function(e) {
console.log(e);
})</code></pre>
</div>
</div>
</li>
</ol>
</div>
<div class="paragraph">
<p>Clearly <code>MathContext</code> &amp; <code>FixedPoint6</code> have a circular dependency upon each other.</p>
</div>
</div>
<div class="sect3 language-js">
<h4 id="sec:Transpile_and_Execute"><a class="anchor" href="#sec:Transpile_and_Execute"></a><a class="link" href="#sec:Transpile_and_Execute">B.5.2. Transpile and Execute</a></h4>
<div class="paragraph">
<p>Transpile the above setup to different formats and execute the code using <code>SystemJS</code> module loader :</p>
</div>
<div class="sect4">
<h5 id="sec:Module_Format___AMD"><a class="anchor" href="#sec:Module_Format___AMD"></a><a class="link" href="#sec:Module_Format___AMD">B.5.2.1. Module Format = AMD</a></h5>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Compile the previous set up using <code>babel</code> as the transpiler with <code>AMD</code> modules :</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>babel src -w --out-dir build --modules amd</code></pre>
</div>
</div>
</li>
<li>
<p>Run the transpiled <code>test.js</code> with :</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>node runner.js</code></pre>
</div>
</div>
</li>
<li>
<p>The execution would fail with an error like the following :</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>Error: _FixedPoint62.default.getQuotient is not a function</code></pre>
</div>
</div>
</li>
</ol>
</div>
</div>
<div class="sect4">
<h5 id="sec:Module_Format___CommonJS"><a class="anchor" href="#sec:Module_Format___CommonJS"></a><a class="link" href="#sec:Module_Format___CommonJS">B.5.2.2. Module Format = CommonJS</a></h5>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Compile the previous set up using <code>babel</code> as the transpiler with <code>CommonJS</code> modules :</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>babel src -w --out-dir build --modules common</code></pre>
</div>
</div>
</li>
<li>
<p>Run the transpiled <code>test.js</code> with :</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>node runner.js</code></pre>
</div>
</div>
</li>
<li>
<p>The execution is successful and logs the following results :</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>{ value: 20.5 } { value: 10 } { value: 2 }</code></pre>
</div>
</div>
</li>
</ol>
</div>
</div>
<div class="sect4">
<h5 id="sec:Module_Format___SystemJS"><a class="anchor" href="#sec:Module_Format___SystemJS"></a><a class="link" href="#sec:Module_Format___SystemJS">B.5.2.3. Module Format = SystemJS</a></h5>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Compile the previous set up using <code>babel</code> as the transpiler with <code>SystemJS</code> modules :</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>babel src -w --out-dir build --modules system</code></pre>
</div>
</div>
</li>
<li>
<p>Run the transpiled <code>test.js</code> with :</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>node runner.js</code></pre>
</div>
</div>
</li>
<li>
<p>The execution is successful and logs the following results :</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>{ value: 20.5 } { value: 10 } { value: 2 }</code></pre>
</div>
</div>
</li>
</ol>
</div>
</div>
</div>
<div class="sect3">
<h4 id="sec:Conclusion"><a class="anchor" href="#sec:Conclusion"></a><a class="link" href="#sec:Conclusion">B.5.3. Conclusion</a></h4>
<div class="paragraph">
<p>As observed, the test is executed successfully with <code>CommonJS</code> &amp; <code>SystemJS</code> module formats. It however fails with <code>AMD</code> format (due to the circular dependency).</p>
</div>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="system.register-as-transpilation-target"><a class="anchor" href="#system.register-as-transpilation-target"></a><a class="link" href="#system.register-as-transpilation-target">B.6. System.register as transpilation target</a></h3>
<div class="paragraph">
<p>In order to integrate <code>SystemJS</code> as the module loader, the recommended module format is <code>System.register</code>. This section serves as a guide (&amp; implementation hint) to transpile N4JS modules with <code>System.register</code> as the module format.</p>
</div>
<div class="sect3">
<h4 id="sec:Introduction"><a class="anchor" href="#sec:Introduction"></a><a class="link" href="#sec:Introduction">B.6.1. Introduction</a></h4>
<div class="paragraph">
<p>This format is best explained from its <a href="https://github.com/ModuleLoader/es6-module-loader/blob/master/docs/system-register.md">documentation</a> :</p>
</div>
<div class="quoteblock">
<blockquote>
<div class="paragraph">
<p>System.register can be considered as a new module format designed to support the exact semantics of ES6 modules within ES5. It is a format that was developed out of collaboration and is supported as a module output in Traceur (as instantiate), Babel and TypeScript (as system). All dynamic binding and circular reference behaviors supported by ES6 modules are supported by this format. In this way it acts as a safe and comprehensive target format for the polyfill path into ES6 modules.</p>
</div>
<div class="paragraph">
<p>To run the format, a suitable loader implementation needs to be used that understands how to execute it. Currently these include SystemJS, SystemJS Self-Executing Bundles and ES6 Micro Loader. The ES6 Module Loader polyfill also uses this format internally when transpiling and executing ES6.</p>
</div>
</blockquote>
</div>
<div class="paragraph">
<p>The <code>System.register</code> format is not very well documented. However, this format is supported by all major transpilers out there i.e. <code>BabelJS</code>, <code>Traceur</code> &amp; <code>TypeScript</code> transpilers. In fact, the primary resource of this documentation has been the outputs generated by these transpilers.</p>
</div>
<div class="sect4">
<h5 id="sec:External_Transpilers"><a class="anchor" href="#sec:External_Transpilers"></a><a class="link" href="#sec:External_Transpilers">B.6.1.1. External Transpilers</a></h5>
<div class="paragraph">
<p>In order to follow along, it will be best to try out different ES6 syntax being transpiled to <code>System.register</code> format by these transpilers.</p>
</div>
<div class="paragraph">
<p>The following instructions will be useful :</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Transpile with Traceur</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-bash" data-lang="bash">traceur --dir &lt;SOURCE_DIR&gt; &lt;OUTPUT_DIR&gt; --experimental --modules=instantiate</code></pre>
</div>
</div>
</li>
<li>
<p>Transpile with Babel</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-bash" data-lang="bash">babel &lt;SOURCE_DIR&gt; --out-dir &lt;OUTPUT_DIR&gt; --modules system</code></pre>
</div>
</div>
</li>
<li>
<p>Transpile with TypeScript</p>
<div class="paragraph">
<p>Create a file by the name of <code>tsconfig.json</code> in the project folder, with the following contents :</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-json" data-lang="json">{
"compilerOptions": {
"module": "system",
"target": "ES5",
"outDir": &lt;OUTPUT_DIR&gt;,
"rootDir": &lt;SOURCE_DIR&gt;
}
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>Then transpile with :</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-javascript" data-lang="javascript">tsc</code></pre>
</div>
</div>
</li>
</ol>
</div>
</div>
<div class="sect4">
<h5 id="sec:Example_of_a_System_register_module"><a class="anchor" href="#sec:Example_of_a_System_register_module"></a><a class="link" href="#sec:Example_of_a_System_register_module">B.6.1.2. Example of a System.register module</a></h5>
<div class="paragraph">
<p>For the following ES6 code :</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-javascript" data-lang="javascript">import { p as q } from './dep';
var s = 'local';
export function func() {
return q;
}
export class C {}</code></pre>
</div>
</div>
<div class="paragraph">
<p>The <code>Babel</code> transpiler generates the following code (w/ <code>System.register</code> format):</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-javascript" data-lang="javascript">System.register(['./dep'], function (_export) {
'use strict';
var q, s, C;
_export('func', func);
function _classCallCheck(instance, Constructor) { .. }
function func() {
return q;
}
return {
setters: [function (_dep) {
q = _dep.p;
}],
execute: function () {
s = 'local';
C = function C() {
_classCallCheck(this, C);
};
_export('C', C);
}
};
});</code></pre>
</div>
</div>
</div>
</div>
<div class="sect3">
<h4 id="sec:Structure_of_a_System_register_module"><a class="anchor" href="#sec:Structure_of_a_System_register_module"></a><a class="link" href="#sec:Structure_of_a_System_register_module">B.6.2. Structure of a System.register module</a></h4>
<div class="paragraph">
<p>Broadly speaking, a <code>System.register</code> module has the following structure :</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-javascript" data-lang="javascript">System.register(&lt;&lt;DEPENDENCIES ARRAY&gt;&gt;, function(&lt;&lt;exportFn&gt;&gt;) {
&lt;&lt;DECLARATION SCOPE&gt;&gt;
return {
setters: &lt;&lt;SETTERS ARRAY&gt;&gt;,
execute: function() {
&lt;&lt;EXECUTABLES&gt;&gt;
}
};
});</code></pre>
</div>
</div>
<div class="paragraph">
<p>Highlights :</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>System.register(&#8230;&#8203;)</code> is called with 2 arguments :</p>
<div class="ulist">
<ul>
<li>
<p><code>&lt;&lt;DEPENDENCIES ARRAY&gt;&gt;</code> : an array of dependencies (similar to <code>AMD</code>) extracted from the <code>ES6 import</code> statements.</p>
</li>
<li>
<p>a function (<code>FN</code>) :</p>
<div class="ulist">
<ul>
<li>
<p>accepts a parameter called <code>&lt;&lt;exportFn&gt;&gt;</code>. This <code>&lt;&lt;exportFn&gt;&gt;</code> (provided by <code>SystemJS</code>) keeps a track of all the exports of this module &amp; will inform all the importing modules of any changes.</p>
</li>
<li>
<p>contains a <code>&lt;&lt;DECLARATION SCOPE&gt;&gt;</code> where all the functions and variables (from the source code) get hoisted to.</p>
</li>
<li>
<p>returns an object with 2 properties :</p>
<div class="ulist">
<ul>
<li>
<p><code>setters</code> : The <code>&lt;&lt;SETTERS ARRAY&gt;&gt;</code> is simply an array of functions. Each of these functions represents the imported-bindings from the <code>&lt;&lt;DEPENDENCIES ARRAY&gt;&gt;</code>. The <code>&lt;&lt;SETTERS ARRAY&gt;&gt;</code> and <code>&lt;&lt;DEPENDENCIES ARRAY&gt;&gt;</code> follow the same order.</p>
</li>
<li>
<p><code>execute</code> : <code>&lt;&lt;EXECUTABLES&gt;&gt;</code> is the rest of the body of the source code.</p>
</li>
</ul>
</div>
</li>
</ul>
</div>
</li>
</ul>
</div>
</li>
</ul>
</div>
</div>
<div class="sect3">
<h4 id="_transpilation-hints"><a class="anchor" href="#_transpilation-hints"></a><a class="link" href="#_transpilation-hints">B.6.3. Transpilation Hints</a></h4>
<div class="paragraph">
<p>By observing the existing transpilers’ output, this sub-section provides insights into the process of transpiling to this format :</p>
</div>
<div class="sect4">
<h5 id="sec:Handling_Imports"><a class="anchor" href="#sec:Handling_Imports"></a><a class="link" href="#sec:Handling_Imports">B.6.3.1. Handling Imports</a></h5>
<div class="paragraph">
<p>The following are ES6 code snippets with some <code>import</code> statements :</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Simple Import Statement</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-javascript" data-lang="javascript">import {b1} from 'B';</code></pre>
</div>
</div>
<div class="paragraph">
<p>A valid <code>System.register</code> output for this snippet would look like :</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>System.register(['B'], function (&lt;&lt;exportFn&gt;&gt;) { //(1.)
var b1; //(2.)
return {
//(3.)
setters: [function (_B) {
b1 = _B.b1; //(4.)
}],
execute: function () {}
};
});</code></pre>
</div>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
highlights
</td>
<td class="hdlist2">
<div class="olist arabic">
<ol class="arabic">
<li>
<p>The <code>&lt;&lt;DEPENDENCIES ARRAY&gt;&gt;</code> is just <code>[’B’]</code>.</p>
</li>
<li>
<p>The <code>&lt;&lt;DECLARATION SCOPE&gt;&gt;</code> simply declares the imported binding <code>v1</code> as a variable.</p>
</li>
<li>
<p>The <code>&lt;&lt;SETTERS ARRAY&gt;&gt;</code> has 1 function. This function corresponds to the single dependency (<code>’B’</code>) from (1.)</p>
</li>
<li>
<p>The setter function accepts one argument (the exported object from <code>’B’</code> as <code>_B</code> . It then sets the local binding (i.e. local variable <code>v1</code>) to <code>_B.b1</code>.</p>
</li>
</ol>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
Takeaway
</td>
<td class="hdlist2">
<div class="ulist">
<ul>
<li>
<p>An <code>import</code> statement is broken down into <code>&lt;&lt;DEPENDENCIES ARRAY&gt;&gt;</code> &amp; <code>&lt;&lt;SETTERS ARRAY&gt;&gt;</code>.</p>
</li>
<li>
<p>Whenever the value of <code>b1</code> inside <code>B</code> is changed, <code>SystemJS</code> will execute the corresponding <code>setter function</code> in this module i.e. the 1st function in this case.</p>
</li>
</ul>
</div>
</td>
</tr>
</table>
</div>
</li>
<li>
<p>Multiple Import Statements</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import { a1 as a0 } from 'A';
import {b1} from 'B';
import { c1 as c0 } from 'C';
import {b2, b3} from 'B';
import {default} from 'C';
import {a2} from 'A';</code></pre>
</div>
</div>
<div class="paragraph">
<p>A valid <code>System.register</code> output for this snippet would look like :</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>System.register(['A', 'B', 'C'], function (&lt;&lt;exportFn&gt;&gt;) { //(1.)
var a0, a2, b1, b2, b3, c0, default; //(2.)
return {
//(3.)
setters: [function (_A) {
a0 = _A.a1; //(4.1.)
a2 = _A.a2; //(4.2.)
}, function (_B) {
b1 = _B.b1;
b2 = _B.b2;
b3 = _B.b3;
}, function (_C) {
c0 = _C.c1;
default = _C['default'];
}],
execute: function () {}
};
});</code></pre>
</div>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
highlights
</td>
<td class="hdlist2">
<div class="olist arabic">
<ol class="arabic">
<li>
<p>The <code>&lt;&lt;DEPENDENCIES ARRAY&gt;&gt;</code> is now a unique array <code>[’A’, ’B’, ’C’]</code>. Note that there are no duplicates.</p>
</li>
<li>
<p>The <code>&lt;&lt;DECLARATION SCOPE&gt;&gt;</code> simply declares all the imported bindings as variables.</p>
</li>
<li>
<p>The <code>&lt;&lt;SETTERS ARRAY&gt;&gt;</code> now has 3 functions. These 3 functions match the ordering of the <code>&lt;&lt;DEPENDENCIES ARRAY&gt;&gt;</code>.</p>
</li>
<li>
<p>The setter function accepts one argument (the exported object from the dependency) It then sets the local bindings (i.e. local variables) from the exported value of the dependency.</p>
</li>
</ol>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
Takeaway
</td>
<td class="hdlist2">
<div class="ulist">
<ul>
<li>
<p>Whenever an exported value from <code>A</code> is changed, <code>SystemJS</code> will execute the first <code>setter function</code> in this module.</p>
</li>
<li>
<p>Whenever an exported value from <code>B</code> is changed, <code>SystemJS</code> will execute the second <code>setter function</code> in this module.</p>
</li>
<li>
<p>Whenever an exported value from <code>C</code> is changed, <code>SystemJS</code> will execute the third <code>setter function</code> in this module.</p>
</li>
</ul>
</div>
</td>
</tr>
</table>
</div>
</li>
</ol>
</div>
</div>
<div class="sect4">
<h5 id="sec:__exportFn__"><a class="anchor" href="#sec:__exportFn__"></a><a class="link" href="#sec:__exportFn__">B.6.3.2. &lt;&lt;exportFn&gt;&gt;</a></h5>
<div class="paragraph">
<p>Before moving on to handling exports, let’s focus on the SystemJS provided <code>&lt;&lt;exportFn&gt;&gt;</code>.</p>
</div>
<div class="paragraph">
<p>This function looks similar to the following :</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>function (name, value) { //(1.)
module.locked = true;
if (typeof name == 'object') {
for (var p in name)
exports[p] = name[p]; //(2.1.)
}
else {
exports[name] = value; //(2.2.)
}
for (var i = 0, l = module.importers.length; i &lt; l; i++) {
var importerModule = module.importers[i];
if (!importerModule.locked) {
var importerIndex = indexOf.call(importerModule.dependencies, module);
importerModule.setters[importerIndex](exports); //(3.)
}
}
module.locked = false;
return value; //(4.)
}</code></pre>
</div>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
highlights
</td>
<td class="hdlist2">
<div class="olist arabic">
<ol class="arabic">
<li>
<p>The <code>&lt;&lt;exportFn&gt;&gt;</code> takes 2 arguments : <code>name</code> &amp; <code>value</code>.</p>
</li>
<li>
<p>It maintains an <code>exports</code> object with <code>name</code> &amp; <code>value</code>.</p>
</li>
<li>
<p>For every module which imports the current module, it executes the corresponding <code>setter function</code>.</p>
</li>
<li>
<p>It returns the <code>value</code>.</p>
</li>
</ol>
</div>
</td>
</tr>
</table>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
Takeaway
</td>
<td class="hdlist2">
<p>This <code>&lt;&lt;exportFn&gt;&gt;</code> is responsible for pushing the changes from a module to every importing module thereby implementing the live binding.</p>
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="sec:Handling_Exports"><a class="anchor" href="#sec:Handling_Exports"></a><a class="link" href="#sec:Handling_Exports">B.6.3.3. Handling Exports</a></h5>
<div class="paragraph">
<p>Now let’s focus on handling <code>export</code> statements.</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Simple Exports Statement</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>export var v =1;
export function f(){}</code></pre>
</div>
</div>
<div class="paragraph">
<p>A valid <code>System.register</code> output for this snippet would look like :</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>System.register([], function (_export) { //(1.)
//(2.)
var v;
function f() {}
_export("f", f); //(4.1)
return {
setters: [],
//(3.)
execute: function () {
v = 1; //(3.1.)
_export("v", v); //(4.2.)
}
};
});</code></pre>
</div>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
highlights
</td>
<td class="hdlist2">
<div class="olist arabic">
<ol class="arabic">
<li>
<p>The <code>&lt;&lt;exportFn&gt;&gt;</code> is named to as <code>_export</code>. (This is an implementation decision by Babel.) The name should be unique to not conflict with any user-defined variable/function names.</p>
</li>
<li>
<p>The <code>&lt;&lt;DECLARATION SCOPE&gt;&gt;</code> hoists the exported variable <code>v</code> and the function <code>f</code>.</p>
</li>
<li>
<p>The <code>&lt;&lt;EXECUTABLES&gt;&gt;</code> zone now contains the executable code from the source module.</p>
</li>
<li>
<p>Initialise the variable <code>v1</code> with the value extracted from the source. This essentially is the executable part of the module.</p>
</li>
<li>
<p>The <code>export</code> function expression results in a call to the <code>_exports</code> function as: <code>_export(f, f)</code></p>
</li>
<li>
<p>The <code>export</code> statement results in a call to the <code>_export</code> function as: <code>_export(v, v)</code></p>
</li>
</ol>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
Takeaway
</td>
<td class="hdlist2">
<div class="ulist">
<ul>
<li>
<p>The module’s exports statements are separated from the hoistable and executable statements.</p>
</li>
<li>
<p>All the exported bindings are tracked by wrapping them inside the <code>&lt;&lt;exportFn&gt;&gt;</code>.</p>
</li>
</ul>
</div>
</td>
</tr>
</table>
</div>
</li>
<li>
<p>Tracking Exported Bindings</p>
<div class="paragraph">
<p>To maintain live bindings, <code>SystemJS</code> needs to track any changes to exported bindings in order to call the <code>setter</code> functions of importing modules. Let’s look at an example for that :</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>export var v1 = 1;
export var v2 = 2;
export var v3 = 3;
export function f() {}
v1++; //(1.)
++v2; //(2.)
v3 += 5; //(3.)
f = null; //(4.)</code></pre>
</div>
</div>
<div class="paragraph">
<p><code>Babel</code> output for this snippet looks like :</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>System.register([], function (_export) {
var v1, v2, v3;
_export("f", f);
function f() {}
return {
setters: [],
execute: function () {
v1 = 1;
_export("v1", v1);
v2 = 2;
_export("v2", v2);
v3 = 3;
_export("v3", v3);
_export("v1", v1 += 1); //(1.)
_export("v2", v2 += 1); //(2.)
_export("v3", v3 += 5); //(3.)
_export("f", f = null); //(4.)
}
};
});</code></pre>
</div>
</div>
<div class="paragraph">
<p><code>Traceur</code> output for this snippet looks like :</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>System.register([], function($__export) {
var v1, v2, v3;
function f() {}
$__export("f", f);
return {
setters: [],
execute: function() {
v1 = 1;
$__export("v1", v1);
v2 = 2;
$__export("v2", v2);
v3 = 3;
$__export("v3", v3);
($__export("v1", v1 + 1), v1++); //(1.)
$__export("v2", ++v2); //(2.)
$__export("v3", v3 += 5); //(3.)
$__export("f", f = null); //(4.)
}
};
});</code></pre>
</div>
</div>
<div class="paragraph">
<p><code>TypeScript</code> output for this snippet looks like :</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>System.register([], function(exports_1) {
var v1, v2, v3;
function f() { }
exports_1("f", f);
return {
setters:[],
execute: function() {
exports_1("v1", v1 = 1);
exports_1("v2", v2 = 2);
exports_1("v3", v3 = 3);
(exports_1("v1", ++v1) - 1); //(1.)
exports_1("v2", ++v2); //(2.)
exports_1("v3", v3 += 5); //(3.)
f = null; //(4.)
}
}
});</code></pre>
</div>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
highlights
</td>
<td class="hdlist2">
<div class="ulist">
<ul>
<li>
<p>The re-assignment of <code>v1, v2, v3 and f</code> is wrapped inside a call to the <code>&lt;&lt;exportFn&gt;&gt;</code> with the updated value.</p>
</li>
</ul>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
Takeaway
</td>
<td class="hdlist2">
<div class="ulist">
<ul>
<li>
<p>While transpiling we need to detect if any exported binding is reassigned. In that case invoke the <code>&lt;&lt;exportFn&gt;&gt;</code> immediately with the new value.</p>
</li>
<li>
<p>Different transpilers perform different optimization tricks, which may be worth looking at.</p>
</li>
</ul>
</div>
</td>
</tr>
</table>
</div>
</li>
<li>
<p>Exporting a Class extending an imported Class.</p>
<div class="paragraph">
<p>Let’s look at the following class declaration :</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import {A} from "A"; //&lt;-- import class A
export class C extends A {}</code></pre>
</div>
</div>
<div class="paragraph">
<p><code>Babel</code> output for this snippet looks like :</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>System.register(["A"], function (_export) {
var A, C;
var _get = function get(_x, _x2, _x3) { ... };
function _classCallCheck(instance, Constructor) { ... }
function _inherits(subClass, superClass) { ... }
return {
setters: [function (_A2) {
A = _A2.A;
}],
execute: function () { //(1.)
C = (function (_A) {
_inherits(C, _A);
function C() {
_classCallCheck(this, C);
_get(Object.getPrototypeOf(C.prototype), "constructor", this).apply(this, arguments);
}
return C;
})(A);
_export("C", C);
}
};
});</code></pre>
</div>
</div>
<div class="paragraph">
<p><code>Traceur</code> output for this snippet looks like :</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>System.register(["A"], function($__export) {
var A, C;
return {
setters: [function($__m) {
A = $__m.A;
}],
execute: function() { //(1.)
C = $traceurRuntime.initTailRecursiveFunction(function($__super) {
return $traceurRuntime.call(function($__super) {
function C() {
$traceurRuntime.superConstructor(C).apply(this, arguments);
}
return $traceurRuntime.continuation($traceurRuntime.createClass, $traceurRuntime, [C, {}, {}, $__super]);
}, this, arguments);
})(A);
$__export("C", C);
}
};
});</code></pre>
</div>
</div>
<div class="paragraph">
<p><code>TypeScript</code> output for this snippet looks like :</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>System.register(["A"], function(exports_1) {
var __extends = function(){ ... }
var A_1;
var C;
return {
setters:[
function (A_1_1) {
A_1 = A_1_1;
}],
execute: function() { //(1.)
C = (function (_super) {
__extends(C, _super);
function C() {
_super.apply(this, arguments);
}
return C;
})(A_1.A);
exports_1("C", C);
}
}
});</code></pre>
</div>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
highlights
</td>
<td class="hdlist2">
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Notice how the construction of class <code>C</code> has now been deferred to the <code>&lt;&lt;EXECUTABLES&gt;&gt;</code> zone. It is because <code>C</code> depends on <code>A</code> being imported first.</p>
</li>
</ol>
</div>
</td>
</tr>
<tr>
<td class="hdlist1">
Takeaway
</td>
<td class="hdlist2">
<div class="ulist">
<ul>
<li>
<p>The <code>&lt;&lt;DECLARATION SCOPE&gt;&gt;</code> is for hoisting only independent entities i.e. the ones that do not depend upon any imports. Everything else is moved to the <code>&lt;&lt;EXECUTABLES&gt;&gt;</code> region.</p>
</li>
</ul>
</div>
</td>
</tr>
</table>
</div>
</li>
</ol>
</div>
</div>
</div>
<div class="sect3">
<h4 id="sec:Examples_w__Circular_Dependencies"><a class="anchor" href="#sec:Examples_w__Circular_Dependencies"></a><a class="link" href="#sec:Examples_w__Circular_Dependencies">B.6.4. Examples w/ Circular Dependencies</a></h4>
<div class="paragraph">
<p>This section focuses on circular dependencies. The goal is to see how the transpiled output looks like and if the execution is possible.</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="paragraph">
<p>Source files:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import B from "B";
export default class A {}
A.b = new B(); //&lt;---</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import A from "A";
export default class B {}
B.a = new A(); //&lt;---</code></pre>
</div>
</div>
<div class="paragraph">
<p>Transpiled Outputs (w/ Babel):<br></p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>System.register(["B"], function (_export) {
"use strict";
var B, A;
function _classCallCheck(instance, Constructor) {...}
return {
setters: [function (_B) {
B = _B["default"];
}],
execute: function () {
A = function A() {
_classCallCheck(this, A);
};
_export("default", A);
A.b = new B();
}
};
});</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>System.register(["A"], function (_export) {
"use strict";
var A, B;
function _classCallCheck(instance, Constructor) {...}
return {
setters: [function (_A) {
A = _A["default"];
}],
execute: function () {
B = function B() {
_classCallCheck(this, B);
};
_export("default", B);
B.a = new A();
}
};
});</code></pre>
</div>
</div>
<div class="paragraph">
<p>Execution Result<br></p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>var System = require('systemjs');
System.import('A', 'B').then(function(resp) {
var a = new A();
var b = new B();
}).catch(function(e) {
console.log(e);
});</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>Babel : [Error: undefined is not a function]</code></pre>
</div>
</div>
</div>
</div>
<div class="exampleblock">
<div class="content">
<div class="paragraph">
<p>Source files:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import {B} from "B";
export class A extends B{}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import {A} from "A";
export class B{}
class C extends A{}</code></pre>
</div>
</div>
<div class="paragraph">
<p>Transpiled Outputs (w/ Babel) :<br></p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>System.register(["B"], function (_export) {
"use strict";
var B, A;
var _get = function get(_x, _x2, _x3) { ... };
function _classCallCheck(instance, Constructor) { ... }
function _inherits(subClass, superClass) {...}
return {
setters: [function (_B2) {
B = _B2.B;
}],
execute: function () {
A = (function (_B) {
_inherits(A, _B);
function A() {
_classCallCheck(this, A);
_get(Object.getPrototypeOf(A.prototype), "constructor", this).apply(this, arguments);
}
return A;
})(B);
_export("A", A);
}
};
});</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>System.register(["A"], function (_export) {
"use strict";
var A, B, C;
var _get = function get(_x, _x2, _x3) { ... };
function _inherits(subClass, superClass) { ... }
function _classCallCheck(instance, Constructor) { ... }
return {
setters: [function (_A2) {
A = _A2.A;
}],
execute: function () {
B = function B() {
_classCallCheck(this, B);
};
_export("B", B);
C = (function (_A) {
_inherits(C, _A);
function C() {
_classCallCheck(this, C);
_get(Object.getPrototypeOf(C.prototype), "constructor", this).apply(this, arguments);
}
return C;
})(A);
}
};
});</code></pre>
</div>
</div>
<div class="paragraph">
<p>Execution Result<br></p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>var System = require('systemjs');
System.import('A','B').then(function(resp) {
var a = new A();
}).catch(function(e) {
console.log(e);
});</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>TypeScript : [Error: Cannot read property 'prototype' of undefined]
Babel : [Error: Super expression must either be null or a function, not undefined]</code></pre>
</div>
</div>
</div>
</div>
<div class="exampleblock">
<div class="content">
<div class="paragraph">
<p>Source files:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import B from "B";
class A extends B {}
export default class X {}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import X from "A";
export default class B {}
class Y extends X {}</code></pre>
</div>
</div>
<div class="paragraph">
<p>Transpiled Outputs (w/ Babel):<br></p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>System.register(["B"], function (_export) {
"use strict";
var B, A, X;
var _get = function get(_x, _x2, _x3) { ... };
function _classCallCheck(instance, Constructor) { ... }
function _inherits(subClass, superClass) { ... }
return {
setters: [function (_B2) {
B = _B2["default"];
}],
execute: function () {
A = (function (_B) {
_inherits(A, _B);
function A() {
_classCallCheck(this, A);
_get(Object.getPrototypeOf(A.prototype), "constructor", this).apply(this, arguments);
}
return A;
})(B);
X = function X() {
_classCallCheck(this, X);
};
_export("default", X);
}
};
});</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>System.register(["A"], function (_export) {
"use strict";
var X, B, Y;
var _get = function get(_x, _x2, _x3) { ... };
function _inherits(subClass, superClass) { ... }
function _classCallCheck(instance, Constructor) { ... }
return {
setters: [function (_A) {
X = _A["default"];
}],
execute: function () {
B = function B() {
_classCallCheck(this, B);
};
_export("default", B);
Y = (function (_X) {
_inherits(Y, _X);
function Y() {
_classCallCheck(this, Y);
_get(Object.getPrototypeOf(Y.prototype), "constructor", this).apply(this, arguments);
}
return Y;
})(X);
}
};
});</code></pre>
</div>
</div>
<div class="paragraph">
<p>Execution Result<br></p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>var System = require('systemjs');
System.import('A').then(function(resp) {
var a = new A();
}).catch(function(e) {
console.log(e);
});</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>TypeScript : [Error: Cannot read property 'prototype' of undefined]
Babel : [[Error: Super expression must either be null or a function, not undefined]]</code></pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect3">
<h4 id="sec:N4JS_Examples_w__Circular_Dependencies"><a class="anchor" href="#sec:N4JS_Examples_w__Circular_Dependencies"></a><a class="link" href="#sec:N4JS_Examples_w__Circular_Dependencies">B.6.5. N4JS Examples w/ Circular Dependencies</a></h4>
<div class="paragraph">
<p>In order to improve our precision in conversing and discussing about different kinds of circular dependencies, this section provides the most basic examples of different kinds.</p>
</div>
<div class="sect4">
<h5 id="sec:Unresolved_Cyclic_Dependencies"><a class="anchor" href="#sec:Unresolved_Cyclic_Dependencies"></a><a class="link" href="#sec:Unresolved_Cyclic_Dependencies">B.6.5.1. Unresolved Cyclic Dependencies</a></h5>
<div class="paragraph">
<p>Below examples demonstrate cases when cyclic dependency cannot be resolved at all and will cause runtime errors.</p>
</div>
<div class="exampleblock">
<div class="title">Example 5. Circular dependency resolution 1</div>
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import b from "B"
export public var number a = 1;
export public var number a2 = b + 1;</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import a from "A"
export public var number b = a + 1;</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import a2 from "A"
console.log(a2); //&lt;-- should be 3. not NaN.</code></pre>
</div>
</div>
</div>
</div>
<div class="exampleblock">
<div class="title">Example 6. Circular dependency resolution 2</div>
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import B from "B"
export public class A {
static a = B.b + 1;
}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import A from "A"
export public class B {
static b = 1;
}
export public class B2 {
static b2 = A.a;
}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import B2 from "B"
console.log(B2.b2); //should log 2</code></pre>
</div>
</div>
</div>
</div>
<div class="exampleblock">
<div class="title">Example 7. Circular dependency resolution 3</div>
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import B from "B"
export public class A {
B b = new B();
}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import A from "A"
export public class B {
A a = new A();
}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import A from "A"
new A(); // should not cause a runtime error.</code></pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect4">
<h5 id="sec:Variables___Functions"><a class="anchor" href="#sec:Variables___Functions"></a><a class="link" href="#sec:Variables___Functions">B.6.5.2. Examples with Variables &amp; Functions</a></h5>
<div class="exampleblock">
<div class="title">Example 8. Circular dependency resolution 4</div>
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import b_fun from "B"
export public var a2 = b_fun();
export public var a = 1;</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import a from "A"
export public function b_fun() {
return a + 1;
}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import a2 from "A"
console.log(a2); //&lt;-- should be 2. not NaN.</code></pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect4">
<h5 id="sec:Classes"><a class="anchor" href="#sec:Classes"></a><a class="link" href="#sec:Classes">B.6.5.3. Examples with Classes</a></h5>
<div class="exampleblock">
<div class="title">Example 9. Circular dependency resolution 5</div>
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import B from "B"
export public class A {
static a1 = 1;
static a2 = B.b1;
}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import A from "A"
export public class B {
static b1 = A.a1;
}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import A from "A"
console.log(A.a1); //should log 1. not an error.</code></pre>
</div>
</div>
</div>
</div>
<div class="exampleblock">
<div class="title">Example 10. Circular dependency resolution 6</div>
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import B from "B"
export public class A {
static a1 = 1;
static a2 = B.b1;
}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import A from "A"
export public class B {
static b1 = -1;
static b2 = A.a1;
}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import A from "A"
console.log(A.a1);//should log 1. not an error.</code></pre>
</div>
</div>
</div>
</div>
<div class="exampleblock">
<div class="title">Example 11. Circular dependency resolution 7</div>
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import B from "B"
export public class A {
static a = new B();
}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import A from "A"
export public class B {
static b = new A();
}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import A from "A"
new A(); //should succeed.</code></pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect4">
<h5 id="sec:Examples_with_SubClassing"><a class="anchor" href="#sec:Examples_with_SubClassing"></a><a class="link" href="#sec:Examples_with_SubClassing">B.6.5.4. Examples with SubClassing</a></h5>
<div class="exampleblock">
<div class="title">Example 12. Circular dependency resolution 8</div>
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import B from "B"
export public class A {}
export public class C extends B {}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import A from "A"
export public class B extends A{}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import C from "A"
new C();//should succeed.</code></pre>
</div>
</div>
</div>
</div>
<div class="exampleblock">
<div class="title">Example 13. Circular dependency resolution 9</div>
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import B from "B"
export public class A {}
export public class C {
c = new B();
}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import A from "A"
export public class B extends A{}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import C from "A"
new C(); //should succeed.</code></pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect4">
<h5 id="sec:Miscellaneous"><a class="anchor" href="#sec:Miscellaneous"></a><a class="link" href="#sec:Miscellaneous">B.6.5.5. Miscellaneous</a></h5>
<div class="exampleblock">
<div class="title">Example 14. Circular dependency resolution 10</div>
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import B from "B"
export public class A {}
new B();</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import A from "A"
export public class B {}
new A();</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import A from "A"
new A() //should succeed.</code></pre>
</div>
</div>
</div>
</div>
<div class="exampleblock">
<div class="title">Example 15. Circular dependency resolution 11</div>
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import B from "B"
export public class A {}
B.b1;</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import A from "A"
export public class B {
static b1;
}
new A();</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import A from "A"
new A() //should succeed.</code></pre>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="sect3">
<h4 id="_resources"><a class="anchor" href="#_resources"></a><a class="link" href="#_resources">B.6.6. Resources</a></h4>
<div class="paragraph">
<p><a href="https://github.com/ModuleLoader/es6-module-loader/blob/master/docs/system-register.md">Wiki</a></p>
</div>
</div>
</div>
<div class="sect2 language-js">
<h3 id="sec:CommonJS_as_transpilation_target"><a class="anchor" href="#sec:CommonJS_as_transpilation_target"></a><a class="link" href="#sec:CommonJS_as_transpilation_target">B.7. CommonJS as transpilation target</a></h3>
<div class="paragraph">
<p>To provide better compatibility with <code>npm</code> eco-system, we want to transpile <code>N4JS</code> code to <code>CommonJS</code> module format.</p>
</div>
<div class="sect3">
<h4 id="_introduction-2"><a class="anchor" href="#_introduction-2"></a><a class="link" href="#_introduction-2">B.7.1. Introduction</a></h4>
<div class="paragraph">
<p>A sample <code>CommonJS</code> module :</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>var lib1 = require("/lib1"); //&lt;-- require
var lib2 = require("/lib2"); //&lt;-- require
function fn() {
//...something using 'lib1' &amp; 'lib2'
}
exports.usefulFn = fn; //&lt;--exports
exports.uselessValue = 42; //&lt;--exports</code></pre>
</div>
</div>
<div class="paragraph">
<p>The <a href="http://www.commonjs.org/specs/modules/1.0/">CommonJS spec</a> describes the salient features of module format as (quoted verbatim) :</p>
</div>
<div class="quoteblock">
<blockquote>
<div class="paragraph">
<p>Module Context</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>In a module, there is a free variable "require", that is a
function.</p>
<div class="olist loweralpha">
<ol class="loweralpha" type="a">
<li>
<p>The "require" function accepts a module identifier.</p>
</li>
<li>
<p>"require" returns the exported API of the foreign module.</p>
</li>
<li>
<p>If there is a dependency cycle, the foreign module may
not have finished executing at the time it is required by one
of its transitive dependencies; in this case, the object
returned by "require" must contain at least the exports
that the foreign module has prepared before the call to
require that led to the current module&#8217;s execution.</p>
</li>
<li>
<p>If the requested module cannot be returned, "require"
must throw an error.</p>
</li>
</ol>
</div>
</li>
<li>
<p>In a module, there is a free variable called "exports",
that is an object that the module may add its API to as it
executes.</p>
</li>
<li>
<p>modules must use the "exports" object as the only means
of exporting.</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>Module Identifiers</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>A module identifier is a String of "terms" delimited by forward
slashes.</p>
</li>
<li>
<p>A term must be a camelCase identifier, ".", or "..".</p>
</li>
<li>
<p>Module identifiers may not have file-name extensions like ".js".</p>
</li>
<li>
<p>Module identifiers may be "relative" or "top-level". A module
identifier is "relative" if the first term is "." or "..".</p>
</li>
<li>
<p>Top-level identifiers are resolved off the conceptual module
name space root.</p>
</li>
<li>
<p>Relative identifiers are resolved relative to the identifier of
the module in which "require" is written and called.</p>
</li>
</ol>
</div>
</blockquote>
</div>
</div>
<div class="sect3">
<h4 id="sec:Transpilation_Hints"><a class="anchor" href="#sec:Transpilation_Hints"></a><a class="link" href="#sec:Transpilation_Hints">B.7.2. Transpilation Hints</a></h4>
<div class="paragraph">
<p>This section examines how <code>Babel</code> transpiles <code>ES6</code> modules to <code>CommonJS</code> format. By observing the transpiled output from <code>Babel</code>, we can gather insights for transpiling <code>N4JS</code> modules to <code>CommonJS</code> format.</p>
</div>
<div class="sect4">
<h5 id="sec:Import_Statements"><a class="anchor" href="#sec:Import_Statements"></a><a class="link" href="#sec:Import_Statements">B.7.2.1. Import Statements</a></h5>
<div class="exampleblock">
<div class="title">Example 16. Import an entire module (for side effects only)</div>
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import "B";
console.log(B);</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>"use strict";
require("B");
console.log(B);</code></pre>
</div>
</div>
</div>
</div>
<div class="exampleblock">
<div class="title">Example 17. Import single member of a module</div>
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import {b1} from "B";
b1;</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>"use strict";
var _B = require("B");
_B.b1;</code></pre>
</div>
</div>
</div>
</div>
<div class="exampleblock">
<div class="title">Example 18. Import multiple members of a module</div>
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import {b1, b2} from "B";
b1;
b2;</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>"use strict";
var _B = require("B");
_B.b1;
_B.b2;</code></pre>
</div>
</div>
</div>
</div>
<div class="exampleblock">
<div class="title">Example 19. Import a single member of a module w/ an alias</div>
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import {b3 as b4} from "B";
b4 + 1;</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>"use strict";
var _B = require("B");
_B.b3 + 1;</code></pre>
</div>
</div>
</div>
</div>
<div class="exampleblock">
<div class="title">Example 20. Import multiple members of a module w/ aliases</div>
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import {b3 as b4, b5 as b6} from "B";
b4 + 1;
b6 + 1;</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>"use strict";
var _B = require("B");
_B.b3 + 1;
_B.b5 + 1;</code></pre>
</div>
</div>
</div>
</div>
<div class="exampleblock">
<div class="title">Example 21. Import ALL the bindings of a module</div>
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import * as B from "B";
console.log(B);</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>"use strict";
function _interopRequireWildcard(obj) {
//Babel internally tracks ES6 modules using a flag "__esModule".
if (obj &amp;&amp; obj.__esModule) {
return obj;
} else {
//Copy over all the exported members.
var newObj = {};
if (obj != null) {
for (var key in obj) {
if (Object.prototype.hasOwnProperty.call(obj, key)) newObj[key] = obj[key];
}
}
//Set the "default" as the obj itself (ES6 default export)
newObj["default"] = obj;
return newObj;
}
}
var _B = require("B");
var B = _interopRequireWildcard(_B);
console.log(B);</code></pre>
</div>
</div>
</div>
</div>
<div class="exampleblock">
<div class="title">Example 22. Import the default export of a module</div>
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import B from "B";
console.log(B);</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>"use strict";
//For importing a default export,
//Babel checks if the obj is an ES6 module or not.
function _interopRequireDefault(obj) {
return obj &amp;&amp; obj.__esModule ? obj : { "default": obj };
}
var _B = require("B");
var _B2 = _interopRequireDefault(_B);
console.log(_B2["default"]);</code></pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect4">
<h5 id="sec:Export_Statements"><a class="anchor" href="#sec:Export_Statements"></a><a class="link" href="#sec:Export_Statements">B.7.2.2. Export Statements</a></h5>
<div class="exampleblock">
<div class="title">Example 23. Export a member</div>
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>let a = 1;
export {a};</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>"use strict";
//Babel makes a note that this is as an ES6 module.
//This information is later used when this module is imported.
Object.defineProperty(exports, "__esModule", {
value: true
});
var a = 1;
exports.a = a;</code></pre>
</div>
</div>
</div>
</div>
<div class="exampleblock">
<div class="title">Example 24. Export multiple members</div>
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>let a = 1;
let b = true;
export {a, b};</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>"use strict";
Object.defineProperty(exports, "__esModule", {
value: true
});
var a = 1;
var b = true;
exports.a = a;
exports.b = b;</code></pre>
</div>
</div>
</div>
</div>
<div class="exampleblock">
<div class="title">Example 25. Export using alias</div>
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>let a =1;
export {a as b};</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>"use strict";
Object.defineProperty(exports, "__esModule", {
value: true
});
var a = 1;
exports.b = a;</code></pre>
</div>
</div>
</div>
</div>
<div class="exampleblock">
<div class="title">Example 26. Multiple exports using alias</div>
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>let a = 1, b = 2;
export {a as A, b as B};</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>"use strict";
Object.defineProperty(exports, "__esModule", {
value: true
});
var a = 1,
b = 2;
exports.A = a;
exports.B = b;</code></pre>
</div>
</div>
</div>
</div>
<div class="exampleblock">
<div class="title">Example 27. Simple default export</div>
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>export default 42;</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>"use strict";
Object.defineProperty(exports, "__esModule", {
value: true
});
exports["default"] = 42; //&lt;-- default export is treated as a special named export
module.exports = exports["default"]; //&lt;-- IMPORTANT</code></pre>
</div>
</div>
</div>
</div>
<div class="exampleblock">
<div class="title">Example 28. Default export using an alias</div>
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>let x =10;
export {x as default};</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>"use strict";
Object.defineProperty(exports, "__esModule", {
value: true
});
var x = 10;
exports["default"] = x;
module.exports = exports["default"];</code></pre>
</div>
</div>
</div>
</div>
<div class="exampleblock">
<div class="title">Example 29. Default export w/ named export</div>
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>let a = 1;
export {a};
export default 42;</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>"use strict";
Object.defineProperty(exports, "__esModule", {
value: true
});
var a = 1;
exports.a = a;
exports["default"] = 42;</code></pre>
</div>
</div>
</div>
</div>
<div class="exampleblock">
<div class="title">Example 30. Default export a class</div>
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>export default class A {}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>"use strict";
Object.defineProperty(exports, "__esModule", {
value: true
});
function _classCallCheck(...) { ... }
var A = function A() {
_classCallCheck(this, A);
};
exports["default"] = A;
module.exports = exports["default"];</code></pre>
</div>
</div>
</div>
</div>
<div class="exampleblock">
<div class="title">Example 31. Wildcard re-export</div>
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>export * from "A"</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>"use strict";
Object.defineProperty(exports, "__esModule", {
value: true
});
function _interopExportWildcard(obj, defaults) {
var newObj = defaults({}, obj);
delete newObj["default"]; //&lt;-- A module's default export can not be re-exported.
return newObj;
}
function _defaults(obj, defaults) {
var keys = Object.getOwnPropertyNames(defaults);
for (var i = 0; i &lt; keys.length; i++) {
var key = keys[i];
var value = Object.getOwnPropertyDescriptor(defaults, key);
if (value &amp;&amp; value.configurable &amp;&amp; obj[key] === undefined) {
Object.defineProperty(obj, key, value);
}
}
return obj;
}
var _A = require("A");
_defaults(exports, _interopExportWildcard(_A, _defaults));</code></pre>
</div>
</div>
</div>
</div>
<div class="exampleblock">
<div class="title">Example 32. Specific member re-export</div>
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>export {a1, a2} from "A";</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>"use strict";
Object.defineProperty(exports, "__esModule", {
value: true
});
var _A = require("A");
Object.defineProperty(exports, "a1", {
enumerable: true,
get: function get() {
return _A.a1;
}
});
Object.defineProperty(exports, "a2", {
enumerable: true,
get: function get() {
return _A.a2;
}
});</code></pre>
</div>
</div>
</div>
</div>
<div class="exampleblock">
<div class="title">Example 33. Specific member re-export using alias</div>
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>export {a1 as A1, a2 as A2} from "A";</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>"use strict";
Object.defineProperty(exports, "__esModule", {
value: true
});
var _A = require("A");
Object.defineProperty(exports, "A1", {
enumerable: true,
get: function get() {
return _A.a1;
}
});
Object.defineProperty(exports, "A2", {
enumerable: true,
get: function get() {
return _A.a2;
}
});</code></pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect4">
<h5 id="sec:Tracking_Live_Bindings"><a class="anchor" href="#sec:Tracking_Live_Bindings"></a><a class="link" href="#sec:Tracking_Live_Bindings">B.7.2.3. Tracking Live Bindings</a></h5>
<div class="paragraph">
<p>As specified in the section about <code>ES6 Modules</code> (<a href="#sec:ES6_Modules">ES6 Modules</a>), <code>ES6 Modules</code> export live immutable bindings. The following listings demonstrate how <code>Babel</code> achieves this.</p>
</div>
<div class="exampleblock">
<div class="title">Example 34. Tracking Live Binding</div>
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>export var a = 1;
a++;</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>"use strict";
Object.defineProperty(exports, "__esModule", {
value: true
});
var a = 1;
exports.a = a;
exports.a = a += 1; //&lt;-- Exported value is tracked.</code></pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect4">
<h5 id="sec:A_complete_example"><a class="anchor" href="#sec:A_complete_example"></a><a class="link" href="#sec:A_complete_example">B.7.2.4. A complete example</a></h5>
<div class="paragraph">
<p>The following listings present a simple but complete example of ES6 export, import and live-binding concepts. It uses 3 simple <code>ES6 modules</code> called <code>A.js, B.js and Main.js</code>. The modules are listed alongside their <code>CommonJS</code> versions generated by <code>Babel</code>.</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>export var a = 1; //&lt;-- exports a number
export function incA() { //&lt;-- exports a function
a++;
}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>"use strict";
Object.defineProperty(exports, "__esModule", {
value: true
});
exports.incA = incA;
var a = 1;
exports.a = a;
function incA() {
exports.a = a += 1;
}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import {incA} from "./A"; //&lt;-- Imports the function from A.js
export function incB() { //&lt;-- Exports a function that calls the imported function from A.js
incA();
}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>"use strict";
Object.defineProperty(exports, "__esModule", {
value: true
});
exports.incB = incB;
var _A = require("./A");
function incB() {
_A.incA();
}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>import {a} from "./A"; //&lt;-- Imports the exported number from A.js
import {incB} from "./B"; //&lt;-- Imports the exported function from B.js
console.log(a); //&lt;-- Prints "1"
incB(); //&lt;-- This will call the "incA" function of A.js
console.log(a); //&lt;--Prints "2". The imported value "a" is updated.</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>"use strict";
var _A = require("./A");
var _B = require("./B");
console.log(_A.a);
_B.incB();
console.log(_A.a);</code></pre>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="sect3">
<h4 id="_resources-2"><a class="anchor" href="#_resources-2"></a><a class="link" href="#_resources-2">B.7.3. Resources</a></h4>
<div class="paragraph">
<p><a href="http://exploringjs.com/es6/ch_modules.html">Exploring ES6 by Dr. Axel Rauschmayer</a></p>
</div>
<div class="paragraph">
<p><a href="http://www.commonjs.org/specs/modules/1.0/">CommonJS spec</a></p>
</div>
<div class="paragraph">
<p><a href="http://benjamn.github.io/empirenode-2015/">The Importance of import and export</a></p>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="sec:License"><a class="anchor" href="#sec:License"></a><a class="link" href="#sec:License">Appendix C: License</a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>This specification and the accompanying materials is made available
under the terms of the Eclipse Public License v1.0 which accompanies
this distribution, and is available at <a href="http://www.eclipse.org/legal/epl-v10.html" class="bare">http://www.eclipse.org/legal/epl-v10.html</a></p>
</div>
<h3 id="_eclipse-public-license-v-1-0" class="discrete">Eclipse Public License - v 1.0</h3>
<div class="paragraph">
<p>THE ACCOMPANYING PROGRAM IS PROVIDED UNDER THE TERMS OF THIS ECLIPSE
PUBLIC LICENSE (<code>AGREEMENT</code>). ANY USE, REPRODUCTION OR DISTRIBUTION OF
THE PROGRAM CONSTITUTES RECIPIENT’S ACCEPTANCE OF THIS AGREEMENT.</p>
</div>
<h4 id="_1-definitions" class="discrete">1. DEFINITIONS</h4>
<div class="dlist">
<dl>
<dt class="hdlist1"><code>Contribution</code> means: </dt>
<dd>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>in the case of the initial Contributor, the initial code and
documentation distributed under this Agreement, and</p>
</li>
<li>
<p>in the case of each subsequent Contributor:</p>
<div class="olist loweralpha">
<ol class="loweralpha" type="a">
<li>
<p>changes to the Program, and</p>
</li>
<li>
<p>additions to the Program;</p>
<div class="paragraph">
<p>where such changes and/or additions to the Program originate from and
are distributed by that particular Contributor. A Contribution
’originates’ from a Contributor if it was added to the Program by such
Contributor itself or anyone acting on such Contributor’s behalf.
Contributions do not include additions to the Program which:</p>
</div>
<div class="olist lowerroman">
<ol class="lowerroman" type="i">
<li>
<p>are separate modules of software distributed in conjunction with the Program
under their own license agreement, and</p>
</li>
<li>
<p>are not derivative works of the Program.</p>
</li>
</ol>
</div>
</li>
</ol>
</div>
</li>
</ol>
</div>
</dd>
<dt class="hdlist1"><code>Contributor</code></dt>
<dd>
<p>means any person or entity that distributes the Program.</p>
</dd>
<dt class="hdlist1"><code>Licensed Patents</code> </dt>
<dd>
<p>mean patent claims licensable by a Contributor
which are necessarily infringed by the use or sale of its Contribution
alone or when combined with the Program.</p>
</dd>
<dt class="hdlist1"><code>Program</code> </dt>
<dd>
<p>means the Contributions distributed in accordance with this
Agreement.</p>
</dd>
<dt class="hdlist1"><code>Recipient</code> </dt>
<dd>
<p>means anyone who receives the Program under this
Agreement, including all Contributors.</p>
</dd>
</dl>
</div>
<h4 id="_2-grant-of-rights" class="discrete">2. GRANT OF RIGHTS</h4>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Subject to the terms of this Agreement, each Contributor hereby
grants Recipient a non-exclusive, worldwide, royalty-free copyright
license to reproduce, prepare derivative works of, publicly display,
publicly perform, distribute and sublicense the Contribution of such
Contributor, if any, and such derivative works, in source code and
object code form.</p>
</li>
<li>
<p>Subject to the terms of this Agreement, each Contributor hereby
grants Recipient a non-exclusive, worldwide, royalty-free patent license
under Licensed Patents to make, use, sell, offer to sell, import and
otherwise transfer the Contribution of such Contributor, if any, in
source code and object code form. This patent license shall apply to the
combination of the Contribution and the Program if, at the time the
Contribution is added by the Contributor, such addition of the
Contribution causes such combination to be covered by the Licensed
Patents. The patent license shall not apply to any other combinations
which include the Contribution. No hardware per se is licensed
hereunder.</p>
</li>
<li>
<p>Recipient understands that although each Contributor grants the
licenses to its Contributions set forth herein, no assurances are
provided by any Contributor that the Program does not infringe the
patent or other intellectual property rights of any other entity. Each
Contributor disclaims any liability to Recipient for claims brought by
any other entity based on infringement of intellectual property rights
or otherwise. As a condition to exercising the rights and licenses
granted hereunder, each Recipient hereby assumes sole responsibility to
secure any other intellectual property rights needed, if any. For
example, if a third party patent license is required to allow Recipient
to distribute the Program, it is Recipient’s responsibility to acquire
that license before distributing the Program.</p>
</li>
<li>
<p>Each Contributor represents that to its knowledge it has sufficient
copyright rights in its Contribution, if any, to grant the copyright
license set forth in this Agreement.</p>
</li>
</ol>
</div>
<h4 id="_3-requirements" class="discrete">3. REQUIREMENTS</h4>
<div class="paragraph">
<p>A Contributor may choose to distribute the Program in object code form
under its own license agreement, provided that:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>it complies with the terms and conditions of this Agreement; and</p>
</li>
<li>
<p>its license agreement:</p>
<div class="olist loweralpha">
<ol class="loweralpha" type="a">
<li>
<p>effectively disclaims on behalf of all Contributors all warranties
and conditions, express and implied, including warranties or conditions
of title and non-infringement, and implied warranties or conditions of
merchantability and fitness for a particular purpose;</p>
</li>
<li>
<p>effectively excludes on behalf of all Contributors all liability for
damages, including direct, indirect, special, incidental and
consequential damages, such as lost profits;</p>
</li>
<li>
<p>states that any provisions which differ from this Agreement are
offered by that Contributor alone and not by any other party; and</p>
</li>
<li>
<p>states that source code for the Program is available from such
Contributor, and informs licensees how to obtain it in a reasonable
manner on or through a medium customarily used for software exchange.</p>
</li>
</ol>
</div>
</li>
</ol>
</div>
<div class="paragraph">
<p>When the Program is made available in source code form:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>it must be made available under this Agreement; and</p>
</li>
<li>
<p>a copy of this Agreement must be included with each copy of the
Program.</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>Contributors may not remove or alter any copyright notices contained
within the Program.</p>
</div>
<div class="paragraph">
<p>Each Contributor must identify itself as the originator of its
Contribution, if any, in a manner that reasonably allows subsequent
Recipients to identify the originator of the Contribution.</p>
</div>
<h4 id="_4-commercial-distribution" class="discrete">4. COMMERCIAL DISTRIBUTION</h4>
<div class="paragraph">
<p>Commercial distributors of software may accept certain responsibilities
with respect to end users, business partners and the like. While this
license is intended to facilitate the commercial use of the Program, the
Contributor who includes the Program in a commercial product offering
should do so in a manner which does not create potential liability for
other Contributors. Therefore, if a Contributor includes the Program in
a commercial product offering, such Contributor (<code>Commercial
Contributor</code>) hereby agrees to defend and indemnify every other
Contributor (<code>Indemnified Contributor</code>) against any losses, damages
and costs (collectively <code>Losses</code>) arising from claims, lawsuits and
other legal actions brought by a third party against the Indemnified
Contributor to the extent caused by the acts or omissions of such
Commercial Contributor in connection with its distribution of the
Program in a commercial product offering. The obligations in this
section do not apply to any claims or Losses relating to any actual or
alleged intellectual property infringement. In order to qualify, an
Indemnified Contributor must: a) promptly notify the Commercial
Contributor in writing of such claim, and b) allow the Commercial
Contributor to control, and cooperate with the Commercial Contributor
in, the defense and any related settlement negotiations. The Indemnified
Contributor may participate in any such claim at its own expense.</p>
</div>
<div class="paragraph">
<p>For example, a Contributor might include the Program in a commercial
product offering, Product X. That Contributor is then a Commercial
Contributor. If that Commercial Contributor then makes performance
claims, or offers warranties related to Product X, those performance
claims and warranties are such Commercial Contributor’s responsibility
alone. Under this section, the Commercial Contributor would have to
defend claims against the other Contributors related to those
performance claims and warranties, and if a court requires any other
Contributor to pay any damages as a result, the Commercial Contributor
must pay those damages.</p>
</div>
<h4 id="_5-no-warranty" class="discrete">5. NO WARRANTY</h4>
<div class="paragraph">
<p>EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, THE PROGRAM IS PROVIDED
ON AN <code>AS IS</code> BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
EITHER EXPRESS OR IMPLIED INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES
OR CONDITIONS OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR
A PARTICULAR PURPOSE. Each Recipient is solely responsible for
determining the appropriateness of using and distributing the Program
and assumes all risks associated with its exercise of rights under this
Agreement , including but not limited to the risks and costs of program
errors, compliance with applicable laws, damage to or loss of data,
programs or equipment, and unavailability or interruption of operations.</p>
</div>
<h4 id="_6-disclaimer-of-liability" class="discrete">6. DISCLAIMER OF LIABILITY</h4>
<div class="paragraph">
<p>EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, NEITHER RECIPIENT NOR
ANY CONTRIBUTORS SHALL HAVE ANY LIABILITY FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING
WITHOUT LIMITATION LOST PROFITS), HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OR
DISTRIBUTION OF THE PROGRAM OR THE EXERCISE OF ANY RIGHTS GRANTED
HEREUNDER, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.</p>
</div>
<h4 id="_7-general" class="discrete">7. GENERAL</h4>
<div class="paragraph">
<p>If any provision of this Agreement is invalid or unenforceable under
applicable law, it shall not affect the validity or enforceability of
the remainder of the terms of this Agreement, and without further action
by the parties hereto, such provision shall be reformed to the minimum
extent necessary to make such provision valid and enforceable.</p>
</div>
<div class="paragraph">
<p>If Recipient institutes patent litigation against any entity (including
a cross-claim or counterclaim in a lawsuit) alleging that the Program
itself (excluding combinations of the Program with other software or
hardware) infringes such Recipient’s patent(s), then such Recipient’s
rights granted under Section 2(b) shall terminate as of the date such
litigation is filed.</p>
</div>
<div class="paragraph">
<p>All Recipient’s rights under this Agreement shall terminate if it fails
to comply with any of the material terms or conditions of this Agreement
and does not cure such failure in a reasonable period of time after
becoming aware of such noncompliance. If all Recipient’s rights under
this Agreement terminate, Recipient agrees to cease use and distribution
of the Program as soon as reasonably practicable. However, Recipient’s
obligations under this Agreement and any licenses granted by Recipient
relating to the Program shall continue and survive.</p>
</div>
<div class="paragraph">
<p>Everyone is permitted to copy and distribute copies of this Agreement,
but in order to avoid inconsistency the Agreement is copyrighted and may
only be modified in the following manner. The Agreement Steward reserves
the right to publish new versions (including revisions) of this
Agreement from time to time. No one other than the Agreement Steward has
the right to modify this Agreement. The Eclipse Foundation is the
initial Agreement Steward. The Eclipse Foundation may assign the
responsibility to serve as the Agreement Steward to a suitable separate
entity. Each new version of the Agreement will be given a distinguishing
version number. The Program (including Contributions) may always be
distributed subject to the version of the Agreement under which it was
received. In addition, after a new version of the Agreement is
published, Contributor may elect to distribute the Program (including
its Contributions) under the new version. Except as expressly stated in
Sections 2(a) and 2(b) above, Recipient receives no rights or licenses
to the intellectual property of any Contributor under this Agreement,
whether expressly, by implication, estoppel or otherwise. All rights in
the Program not expressly granted under this Agreement are reserved.</p>
</div>
<div class="paragraph">
<p>This Agreement is governed by the laws of the State of New York and the
intellectual property laws of the United States of America. No party to
this Agreement will bring a legal action under this Agreement more than
one year after the cause of action arose. Each party waives its rights
to a jury trial in any resulting litigation.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="sec:Acronyms"><a class="anchor" href="#sec:Acronyms"></a><a class="link" href="#sec:Acronyms">Appendix D: Acronyms</a></h2>
<div class="sectionbody">
<table id="AC" class="tableblock frame-all grid-all spread language-bash">
<colgroup>
<col style="width: 12.5%;">
<col style="width: 37.5%;">
<col style="width: 12.5%;">
<col style="width: 37.5%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>CDep</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">Compile-Time Dependency</p></th>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>RDep</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">Run-Time Dependency</p></th>
</tr>
<tr>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>LDep</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">Load-Time Dependency</p></th>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>IDep</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">Initialization-Time Dependency</p></th>
</tr>
<tr>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>EDep</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">Execution-Time Dependency</p></th>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>AC</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">Acceptance Criteria</p></th>
</tr>
<tr>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>ANTLR</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">ANother Tool for Language Recognition</p></th>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>API</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">Application Programming Interface</p></th>
</tr>
<tr>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>AST</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">Abstract Syntax Tree</p></th>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>ASI</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">Automatic Semicolon Insertion</p></th>
</tr>
<tr>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>AST</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">Abstract Syntax Tree</p></th>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>BNF</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">Backus-Naur Form</p></th>
</tr>
<tr>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>CA</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">Content-Assist</p></th>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>CSP</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">Constraint Satisfaction Problem</p></th>
</tr>
<tr>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>CLI</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">Command Line Interface</p></th>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>DOM</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">Document Object Model</p></th>
</tr>
<tr>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>DSL</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">Domain Specific Language</p></th>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>EBNF</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">Extended Backus-Naur Form</p></th>
</tr>
<tr>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>EMF</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">Eclipse Modeling Framework</p></th>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>EPL</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">Eclipse Public License</p></th>
</tr>
<tr>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>FQN</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">Fully Qualified Name</p></th>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>GLB</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">Greatest Lower Bound, also known as <strong>infimum</strong></p></th>
</tr>
<tr>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>GPL</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">GNU General Public License</p></th>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>IDE</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">Integrated Development Environment</p></th>
</tr>
<tr>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>IDL</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">Interface Definition Language</p></th>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>LSP</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">Liskov Substitution Principle</p></th>
</tr>
<tr>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>LUB</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">Least Upper Bound, also known as <strong>supremum</strong></p></th>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>N4JS</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">NumberFour JavaScript</p></th>
</tr>
<tr>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>UI</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">User Interface</p></th>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>UML</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">Unified Modeling Language</p></th>
</tr>
<tr>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>VM</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">Virtual Machine</p></th>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>XML</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">Extensible Markup Language</p></th>
</tr>
<tr>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>XSLT</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock"><a href="#XSL">XSL</a> Transformations</p></th>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>XSL <a id="XSL"></a></code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">Extensible Stylesheet Language</p></th>
</tr>
<tr>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>WYSIWYG</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">What You See Is What You Get</p></th>
<td class="tableblock halign-center valign-top"><p class="tableblock"><code>WLOG</code></p></td>
<th class="tableblock halign-center valign-top"><p class="tableblock">without loss of generality</p></th>
</tr>
</tbody>
</table>
</div>
</div>
<div class="sect1">
<h2 id="_bibliography-and-footnotes"><a class="anchor" href="#_bibliography-and-footnotes"></a><a class="link" href="#_bibliography-and-footnotes">Appendix E: Bibliography and Footnotes</a></h2>
<div class="sectionbody">
<div class="openblock bibliography">
<div class="content">
<div class="paragraph">
<p><a id="N4JSSpec"></a>N4JS Project. (2018). <em>N4JS Language Specification</em>. Retrieved from <a href="https://www.eclipse.org/n4js/spec/N4JSSpec.html" class="bare">https://www.eclipse.org/n4js/spec/N4JSSpec.html</a></p>
</div>
<div class="paragraph">
<p><a id="Xpect"></a><em>Xpect, Project Website</em>. Retrieved from <a href="https://projects.eclipse.org/projects/modeling.xpect" class="bare">https://projects.eclipse.org/projects/modeling.xpect</a></p>
</div>
<div class="paragraph">
<p><a id="RFC8259"></a>Bray, Tim. (2017). <em>RFC 8259: The javascript object notation (json) data interchange format</em>. </p>
</div>
<div class="paragraph">
<p><a id="ECMA404"></a>International, ECMA. (2017). <em>Standard ECMA-404, The JSON Data Interchange Syntax</em>. </p>
</div>
<div class="paragraph">
<p><a id="RFC7158"></a>Bray, Tim. (2014). <em>RFC 7158: The JavaScript Object Notation ({JSON}) Data Interchange Format</em>. </p>
</div>
<div class="paragraph">
<p><a id="Ser18"></a>Seriot, Nicolas. (2018). <em>Parsing JSON is a Minefield</em>. Retrieved from <a href="http://seriot.ch/parsing_json.php" class="bare">http://seriot.ch/parsing_json.php</a></p>
</div>
<div class="paragraph">
<p><a id="ECMA15a"></a>ECMA. (2015). <em>ECMAScript 2015 Language Specification</em>. Retrieved from <a href="http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-262.pdf" class="bare">http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-262.pdf</a></p>
</div>
<div class="paragraph">
<p><a id="WhatWGLoader"></a>WhatWG. <em>Loader: A Collection of Interesting Ideas</em>. Retrieved from <a href="http://whatwg.github.io/loader/" class="bare">http://whatwg.github.io/loader/</a></p>
</div>
</div>
</div>
</div>
</div>
</div>
<div id="footnotes">
<hr>
<div class="footnote" id="_footnote_1">
<a href="#_footnoteref_1">1</a>. This is not yet implemented as of September 2015; types are still stored in a separate cache, the <code>ASTMetaInfoCache</code>.
</div>
<div class="footnote" id="_footnote_2">
<a href="#_footnoteref_2">2</a>. In the future, the top-down order could become more important if inference of <em>expected</em> types is also integrated into post-processing.
</div>
<div class="footnote" id="_footnote_3">
<a href="#_footnoteref_3">3</a>. The <code class="language-n4js">DeferredTypeRef</code> has replaced the old <code class="language-n4js">ComputedTypeRef</code> that had been used until Summer 2015; those were resolved lazily when the type was actually needed (triggered on demand). For a discussion of this change see <a href="#sec:Type_Inference_combined_with_AST_Traversal__Background">Background</a> and in particular <a href="#tab:typeInferenceBeforeAfter">Comparison of inference of type of AST nodes before / after refactoring.</a>.
</div>
<div class="footnote" id="_footnote_4">
<a href="#_footnoteref_4">4</a>. First, according to the build order.
</div>
<div class="footnote" id="_footnote_5">
<a href="#_footnoteref_5">5</a>. These are not really input values but rather values changed during the following invocation of the IBuilderState that need to be carried over from one invocation to the next.
</div>
<div class="footnote" id="_footnote_6">
<a href="#_footnoteref_6">6</a>. Once the build phase has ended, this copied and modified Xtext index will replace the actual state of the builder state and will be persisted on graceful application shutdown.
</div>
<div class="footnote" id="_footnote_7">
<a href="#_footnoteref_7">7</a>. This set of URIs will contain the URIs of all resources that are available in the copied Xtext index but not yet directly processed by the builder in the current build phase. These URIs will later be used as candidates for all resources that might be marked as affected ones and queued by the builder for forthcoming build phases.
</div>
<div class="footnote" id="_footnote_8">
<a href="#_footnoteref_8">8</a>. This set eventually represents all changes that were made during the current build phase. Note that <code>allChanges</code> might contain resource description deltas that do not represent an actual change, it is processed by the builder but the underlying information stored in the user data is still unchanged.
</div>
<div class="footnote" id="_footnote_9">
<a href="#_footnoteref_9">9</a>. Note that deltas for to-be-deleted resources were already added to <code>allDeltas</code> upfront in step <a href="#itm:processDeleted">Process Deleted</a>.
</div>
<div class="footnote" id="_footnote_10">
<a href="#_footnoteref_10">10</a>. This happens through a call to <code class="language-n4js">CurrentDescriptions#register(Delta)</code>
</div>
<div class="footnote" id="_footnote_11">
<a href="#_footnoteref_11">11</a>. Unlike in step <a href="#itm:enqueueAffectedResources">Enqueue Affected Resources</a>, we now use <code>changedDeltas</code> instead of <code>allDeltas</code> as a basis.
</div>
<div class="footnote" id="_footnote_12">
<a href="#_footnoteref_12">12</a>. One could think of an optimization to only register those types that are not just imported or declared, but whose features are really in use. E.g., in one file another type be imported (and even used as type of variable), but non of its member is used. So changes to these members wouldn’t affect the current resources. However this might miss certain cases. E.g., when a method in the super class is removed and now the method with same signature of a consumed role would be used. The method of the role has no been used before, yet must not be ignored. Thus, currently all super classes, roles and interfaces and referenced classes are added as dependency regardless if their members are called.
</div>
<div class="footnote" id="_footnote_13">
<a href="#_footnoteref_13">13</a>. <a href="https://developer.apple.com/library/mac/documentation/Carbon/Reference/WebKit_JavaScriptCore_Ref/_index.html" class="bare">https://developer.apple.com/library/mac/documentation/Carbon/Reference/WebKit_JavaScriptCore_Ref/_index.html</a>
</div>
<div class="footnote" id="_footnote_14">
<a href="#_footnoteref_14">14</a>. but note that most utility methods obtain the transpiler state automatically; so, most of the time, you won’t need to obtain the state yourself.
</div>
<div class="footnote" id="_footnote_15">
<a href="#_footnoteref_15">15</a>. Currently we use our own fork of Xpect <a href="https://github.com/NumberFour/Xpect" class="bare">https://github.com/NumberFour/Xpect</a> and the respective p2-repository <a href="https://numberfour.github.io/Xpect/updatesite/nightly/" class="bare">https://numberfour.github.io/Xpect/updatesite/nightly/</a>
</div>
<div class="footnote" id="_footnote_16">
<a href="#_footnoteref_16">16</a>. version at the time of writing is Xtext 2.12
</div>
<div class="footnote" id="_footnote_17">
<a href="#_footnoteref_17">17</a>. <a href="https://github.com/eclipse/xtext-core/issues/12" class="bare">https://github.com/eclipse/xtext-core/issues/12</a>
</div>
<div class="footnote" id="_footnote_18">
<a href="#_footnoteref_18">18</a>. Calculates a list of external library projects that have to be build and another list of projects that have to be cleaned.
</div>
</div>
<div id="footer">
<div id="footer-text">
Version 0.9<br>
Last updated 2019-08-08 13:15:33 CEST
</div>
</div>
<!-- ************* docinfo-footer *************************************************************** -->
<div class="Grid social" style="color:#d5dfea">
<div class="Cell Cell--2-12 m-Cell--withMargin">
<h2>Quick Links</h2>
<ul>
<li><a href="../downloads.html">Download</a></li>
<li><a href="../userguides/index.html">Documentation</a></li>
<li><a href="https://github.com/eclipse/n4js/">Source</a></li>
<li><a href="https://github.com/eclipse/n4js/issues">Issues</a></li>
</ul>
</div>
<div class="Cell Cell--2-12 m-Cell--withMargin">
<br/><br/>
<ul>
<li><a href="https://www.eclipse.org/forums/index.php/f/365/">Forum</a></li>
<li><a href="http://n4js.blogspot.de/">Blog</a></li>
<li><a href="https://dev.eclipse.org/mailman/listinfo/n4js-dev">Mailing List</a></li>
<li><a href="https://projects.eclipse.org/projects/technology.n4js">Eclipse Project Page</a></li>
<li><a href="https://twitter.com/n4jsdev">Tweets by n4jsdev</a></li>
</ul>
</div>
<div class="Cell Cell--2-12 m-Cell--withMargin">
<br/><br/>
<ul>
<li><a href="http://www.eclipse.org/">Eclipse Home</a></li>
<li><a href="http://www.eclipse.org/legal/privacy.php">Privacy Policy</a></li>
<li><a href="http://www.eclipse.org/legal/termsofuse.php">Terms of Use</a></li>
<li><a href="http://www.eclipse.org/legal/copyright.php">Copyright Agent</a></li>
<li><a href="http://www.eclipse.org/legal/">Legal</a></li>
</ul>
</div>
<div style="clear: both; height: 0; overflow: hidden;"></div>
</div>
<!-- ************* UI Scripts ************* -->
<script type="text/javascript" src="scripts/back-to-top.js"></script>
<script type="text/javascript" src="scripts/treeview.js"></script>
<script type="text/javascript" src="scripts/toc.js"></script>
<!-- ************* Prism.js Syntax Highlighting ************* -->
<script src="scripts/prism.js"></script>
<script type="text/javascript">
// Add the 'toclist' id for search function
$(".toc2 > ul").attr('id', 'toclist');
// Generate a Search input form
$("#toclist > li:first-of-type").before('<input type="text" id="pagesearch" onkeyup="search()" placeholder="Search for section...">');
$("#toclist > li:first-of-type").before('<i id="clear" class="fa fa-times-circle-o"></i>');
$("#clear").click(function(){
$("#pagesearch").val('');
search();
$('.toc2 > ul').treeView('collapseAll');
});
// intialize Treeview.js
$(".toc2 > ul").treeView();
// Initialize Scrollspy
</script>
<!-- ************* docinfo-footer *************************************************************** -->
</body>
</html>