blob: 6e1e07a7ee94220a7a031752ebf44217f11ae925 [file] [log] [blame]
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<!--[if IE]><meta http-equiv="X-UA-Compatible" content="IE=edge"><![endif]-->
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="generator" content="Asciidoctor 1.5.5">
<meta name="author" content="2019-08-08 13:15:33 CEST">
<title>N4JS Design Specification</title>
<link rel="stylesheet" href="styles/spec.min.css">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.6.3/css/font-awesome.min.css">
<!-- ************* docinfo ******************************************************************* -->
<!-- ************* Favicon ************-->
<link rel="icon" href="images/favicon.ico" />
<!-- ************* Back-to-top JQuery ************* -->
<script src="https://code.jquery.com/jquery-1.12.4.js"></script>
<script src="https://code.jquery.com/ui/1.12.0/jquery-ui.js"></script>
<link href="styles/prism.min.css" rel="stylesheet" />
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/MathJax.js?config=TeX-MML-AM_CHTML">
</script>
<!-- ************* Styles ************* -->
<link rel="stylesheet" type="text/css" href="styles/n4jsspec-adoc.css">
<!-- ****************** NavBar ****************** -->
<div id="menubar">
<div class="banner">
<a href="https://www.eclipse.org/n4js/#"><img id="logo" src="images/n4js-logo.png" alt="Eclipse N4JS"></a>
</div>
<ul>
<li><a href="index.html">Index</a></li>
</ul>
</div>
<!-- ************* docinfo ******************************************************************* -->
<style>
.admonitionblock td.icon .icon-todo:before{content:"\f249";color:#f4ee42}
</style>
</head>
<body class="book toc2 toc-left">
<div id="header">
<h1>N4JS Design Specification</h1>
<div class="details">
<span id="author" class="author">2019-08-08 13:15:33 CEST</span><br>
<span id="revnumber">version 0.9</span>
</div>
<div id="toc" class="toc2">
<div id="toctitle">Table of Contents</div>
<ul class="sectlevel1">
<li><a href="#_introduction">1. Introduction</a>
<ul class="sectlevel2">
<li><a href="#notation">1.1. Notation</a></li>
<li><a href="#sec:IDE_Overview">1.2. IDE Components</a>
<ul class="sectlevel3">
<li><a href="#sec:Naming_Conventions">1.2.1. Naming Conventions</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#_eclipse-setup">2. Eclipse Setup</a>
<ul class="sectlevel2">
<li><a href="#_system-requirements">2.1. System Requirements</a></li>
<li><a href="#_contribute">2.2. Contribute</a>
<ul class="sectlevel3">
<li><a href="#_eclipse-installer">2.2.1. Eclipse Installer</a>
<ul class="sectlevel4">
<li><a href="#_changing-the-setup-script">2.2.1.1. Changing the Setup Script</a></li>
</ul>
</li>
<li><a href="#_manual-ide-configuration">2.2.2. Manual IDE Configuration</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#_release-engineering">3. Release Engineering</a>
<ul class="sectlevel2">
<li><a href="#_nightly-build-on-eclipse-infrastructure">3.1. Nightly build on Eclipse infrastructure</a></li>
<li><a href="#_build-the-n4js-ide-from-command-line">3.2. Build the N4JS IDE from command line</a>
<ul class="sectlevel3">
<li><a href="#_publish-maven-tooling-code-org-eclipse-n4js-releng-util-code">3.2.1. Publish maven-tooling <code>org.eclipse.n4js.releng.util</code></a></li>
<li><a href="#sec:test-verdaccio">3.2.2. Test Verdaccio containing n4js-libs</a></li>
<li><a href="#_generation-of-eclipse-help-for-spec-and-design-document">3.2.3. Generation of Eclipse help for spec and design document</a></li>
</ul>
</li>
<li><a href="#_updating-frameworks-and-dependencies">3.3. Updating frameworks and dependencies</a>
<ul class="sectlevel3">
<li><a href="#_update-of-eclipse-emf-xtext-etc">3.3.1. Update of Eclipse, EMF, Xtext, etc.</a></li>
<li><a href="#_update-of-the-embedded-jre">3.3.2. Update of the embedded JRE</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#_tips-and-tricks">4. Tips and Tricks</a>
<ul class="sectlevel2">
<li><a href="#_naming">4.1. Naming</a></li>
<li><a href="#_logging">4.2. Logging</a></li>
<li><a href="#_cancellation-handling">4.3. Cancellation Handling</a></li>
<li><a href="#_caching">4.4. Caching</a></li>
<li><a href="#_dependency-injection">4.5. Dependency Injection</a></li>
<li><a href="#_miscellaneous">4.6. Miscellaneous</a></li>
</ul>
</li>
<li><a href="#_parser">5. Parser</a>
<ul class="sectlevel2">
<li><a href="#sec:Parser_Overview">5.1. Overview</a></li>
<li><a href="#sec:N4JS_Parser">5.2. N4JS Parser</a></li>
<li><a href="#sec:Parser_Generation_Post_Processing">5.3. Parser Generation Post-Processing</a>
<ul class="sectlevel3">
<li><a href="#sec:Automatic_Semicolon_Insertion">5.3.1. Automatic Semicolon Insertion</a>
<ul class="sectlevel4">
<li><a href="#sec:Injected_code_in_the_Antlr_grammar_file">5.3.1.1. Injected code in the Antlr grammar file</a></li>
<li><a href="#sec:Customized_error_recovery">5.3.1.2. Customized error recovery</a></li>
</ul>
</li>
<li><a href="#sec:_No_line_terminator_allowed_here__handling">5.3.2. Async and <code>No line terminator allowed here</code> Handling</a></li>
<li><a href="#sec:Regular_Expression">5.3.3. Regular Expression</a></li>
<li><a href="#sec:Unicode">5.3.4. Unicode</a></li>
<li><a href="#sec:Literals">5.3.5. Literals</a></li>
</ul>
</li>
<li><a href="#sec:Modifiers">5.4. Modifiers</a></li>
<li><a href="#sec:Conflict_Resolutions">5.5. Conflict Resolutions</a>
<ul class="sectlevel3">
<li><a href="#sec:Reserved_Keywords_vs__Identifier_Names">5.5.1. Reserved Keywords vs. Identifier Names</a></li>
<li><a href="#sec:Operators_and_Generics">5.5.2. Operators and Generics</a></li>
</ul>
</li>
<li><a href="#sec:Content_Assist_Parser">5.6. Content-Assist Parser</a></li>
</ul>
</li>
<li><a href="#_type-system">6. Type System</a>
<ul class="sectlevel2">
<li><a href="#sec:Type_Model_and_Grammar">6.1. Type Model and Grammar</a>
<ul class="sectlevel3">
<li><a href="#sec:Type_Model_Overview">6.1.1. Type Model Overview</a></li>
<li><a href="#sec:Built_in_Types">6.1.2. Built-in and Primitive Types</a></li>
<li><a href="#sec:Type_Model_DSL">6.1.3. Type Model DSL</a></li>
</ul>
</li>
<li><a href="#sec:Type_System_Implementation">6.2. Type System Implementation</a></li>
<li><a href="#sec:Type_Inference_combined_with_AST_Traversal">6.3. Type Inference of AST</a>
<ul class="sectlevel3">
<li><a href="#sec:Type_Inference_combined_with_AST_Traversal__Background">6.3.1. Background</a></li>
<li><a href="#sec:Triggering_Type_Inference_of_AST">6.3.2. Triggering</a></li>
<li><a href="#sec:Traversal_Order_During_Type_Inference_of_AST">6.3.3. Traversal Order</a></li>
<li><a href="#sec:Cross_References_During_Type_Inference_of_AST">6.3.4. Cross-References</a></li>
<li><a href="#sec:Function_Accessor_Bodies_During_Type_Inference_of_AST">6.3.5. Function/Accessor Bodies</a></li>
<li><a href="#sec:Poly_Expressions_During_Type_Inference_of_AST">6.3.6. Poly Expressions</a></li>
<li><a href="#sec:Constraint_Solver_used_During_Type_Inference_of_AST">6.3.7. Constraint Solver</a></li>
<li><a href="#sec:Type_Guards_During_Type_Inference_of_AST">6.3.8. Type Guards</a></li>
</ul>
</li>
<li><a href="#sec:Structural_Typing">6.4. Structural Typing</a></li>
</ul>
</li>
<li><a href="#_type-index">7. Type Index</a>
<ul class="sectlevel2">
<li><a href="#sec:Type_Index_Design_Rationale">7.1. Design Rationale</a>
<ul class="sectlevel3">
<li><a href="#sec:Getting_the_Xtext_Index_Content_IResourceDescriptions">7.1.1. Getting the Xtext Index (<code>IResourceDescriptions</code>) Content</a></li>
</ul>
</li>
<li><a href="#sec:Design_Overview">7.2. Design Overview</a></li>
<li><a href="#sec:N4JS_Resource_Load_States">7.3. N4JS Resource Load States</a></li>
<li><a href="#sec:Type_Builder">7.4. Types Builder</a>
<ul class="sectlevel3">
<li><a href="#sec:Type_Inference_not_allowed_in_Type_Builder">7.4.1. Type Inference not allowed in Types Builder</a></li>
<li><a href="#sec:ComputedTypeReferences">7.4.2. Deferred Type References</a></li>
<li><a href="#sec:Use_cases_of_ComputedTypeRef">7.4.3. Use cases of DeferredTypeRef</a></li>
</ul>
</li>
<li><a href="#sec:Incremental_Builder_Overview">7.5. Incremental Builder (Overview)</a>
<ul class="sectlevel3">
<li><a href="#sec:Incremental_Builder_Overview__XtextBuilder">7.5.1. XtextBuilder</a></li>
<li><a href="#sec:Incremental_Builder_Overview__IBuilderState">7.5.2. IBuilderState</a>
<ul class="sectlevel4">
<li><a href="#copy-and-update-xtext-index">7.5.2.1. Copy and Update Xtext Index</a></li>
<li><a href="#build-state-setup-phase">7.5.2.2. Build State Setup Phase</a></li>
<li><a href="#process-queued-uris">7.5.2.3. Process Queued URIs</a></li>
<li><a href="#queueing-affected-resources">7.5.2.4. Queueing Affected Resources</a></li>
</ul>
</li>
<li><a href="#sec:Incremental_Builder_Overview__Example">7.5.3. Example</a></li>
</ul>
</li>
<li><a href="#dirty-state-handling">7.6. Dirty state handling</a>
<ul class="sectlevel3">
<li><a href="#use-case-restoring-types-from-user-data">7.6.1. Use case: Restoring types from user data</a></li>
<li><a href="#use-case-updating-the-xtext-index">7.6.2. Use case: Updating the Xtext index</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#_project-model">8. Project Model</a>
<ul class="sectlevel2">
<li><a href="#sec:Package_json">8.1. Package.json File</a></li>
<li><a href="#_accessing-project-information">8.2. Accessing Project Information</a>
<ul class="sectlevel3">
<li><a href="#sec:IN4JSCore">8.2.1. IN4JSCore</a></li>
<li><a href="#sec:N4JSModel">8.2.2. N4JSModel</a></li>
<li><a href="#sec:N4JSWorkspace">8.2.3. N4JSWorkspace</a></li>
<li><a href="#sec:N4JSProject">8.2.4. N4JSProject</a></li>
<li><a href="#sec:SourceContainer">8.2.5. SourceContainer</a></li>
<li><a href="#sec:N4JSProjectsStateHelper">8.2.6. N4JSProjectsStateHelper</a></li>
</ul>
</li>
<li><a href="#sec:Caching">8.3. Caching</a>
<ul class="sectlevel3">
<li><a href="#_caching-of-externallibraryworkspace">8.3.1. Caching of ExternalLibraryWorkspace</a></li>
<li><a href="#_caching-of-n4jsprojectsstatehelper">8.3.2. Caching of N4JSProjectsStateHelper</a></li>
</ul>
</li>
<li><a href="#sec:WildcardPathFilter">8.4. WildcardPathFilter</a></li>
<li><a href="#sec:ProjectUtils">8.5. ProjectUtils</a></li>
</ul>
</li>
<li><a href="#_binding">9. Binding</a>
<ul class="sectlevel2">
<li><a href="#sec:Binding_Design_Rationale">9.1. Design Rationale</a></li>
<li><a href="#sec:Binding_to_Members">9.2. Binding to Members</a></li>
<li><a href="#sec:Binding_Getter_Setter">9.3. Getter / Setter Binding</a></li>
<li><a href="#chap:Statics">9.4. Static Member Binding</a></li>
<li><a href="#sec:Binding_Enumeration">9.5. Enumeration Literals Binding</a></li>
<li><a href="#sec:Accessibility_of_types_and_members">9.6. Accessibility of types and members</a></li>
<li><a href="#sec:Member_Scope_Example">9.7. Member Scope Example</a></li>
<li><a href="#sec:Scoping_for_Members_of_Composed_Type_Explained">9.8. Scoping for Members of Composed Type (Union/Intersection) Example</a></li>
<li><a href="#sec:Binding_of_Structurally_References_Types">9.9. Structurally References Types</a></li>
<li><a href="#sec:Building">9.10. Building</a>
<ul class="sectlevel3">
<li><a href="#sec:Build_Phases">9.10.1. Build Phases</a></li>
<li><a href="#sec:Build_Scenarios">9.10.2. Build Scenarios</a></li>
<li><a href="#sec:Lazy_linking_problem">9.10.3. Lazy linking problem</a></li>
</ul>
</li>
<li><a href="#sec:Proxies_and_Proxy_Resolution">9.11. Proxies and Proxy Resolution (Overview)</a>
<ul class="sectlevel3">
<li><a href="#xtexts-lazy-linking-proxies">9.11.1. Xtext’s Lazy Linking Proxies</a></li>
<li><a href="#standard-emf-proxies">9.11.2. Standard EMF Proxies</a></li>
<li><a href="#_how-is-proxy-resolution-triggered">9.11.3. How is Proxy Resolution Triggered?</a></li>
<li><a href="#_when-is-proxy-resolution-allowed">9.11.4. When is Proxy Resolution Allowed?</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#_validation">10. Validation</a>
<ul class="sectlevel2">
<li><a href="#sec:validation_overview">10.1. Validation overview</a></li>
<li><a href="#sec:validation_control_flow">10.2. Validation control flow</a></li>
<li><a href="#sec:validation_issue_ids">10.3. Issue IDs and Messages</a></li>
<li><a href="#sec:validation_usage_patterns">10.4. Usage Pattern</a></li>
<li><a href="#sec:validation_links">10.5. Links</a></li>
</ul>
</li>
<li><a href="#_references">11. References</a>
<ul class="sectlevel2">
<li><a href="#sec:usecases">11.1. Use cases</a></li>
<li><a href="#sec:calculation_algorithm">11.2. Calculation algorithm</a>
<ul class="sectlevel3">
<li><a href="#sec:Xtext_default_implementation">11.2.1. Xtext default implementation</a></li>
<li><a href="#sec:N4_implementation">11.2.2. N4JS implementation</a></li>
</ul>
</li>
<li><a href="#sec:PerformanceOfDependencyCalculation">11.3. Performance Of Dependency Calculation</a></li>
<li><a href="#sec:kinds_of_references">11.4. Kinds of references</a>
<ul class="sectlevel3">
<li><a href="#sec:Cross_References_to_be_ignored">11.4.1. Cross References to be ignored</a></li>
<li><a href="#sec:Cross_References_to_be_handled">11.4.2. Cross References to be handled</a></li>
</ul>
</li>
<li><a href="#sec:transitive_dependencies">11.5. Transitive dependencies</a></li>
<li><a href="#sec:find-references">11.6. Find references</a>
<ul class="sectlevel3">
<li><a href="#_background">11.6.1. Background</a></li>
<li><a href="#_how-find-references-work">11.6.2. How Find References Work</a>
<ul class="sectlevel4">
<li><a href="#_step-1-convert-cursor-position-to-declared-element">11.6.2.1. Step 1: Convert Cursor Position to Declared Element</a></li>
<li><a href="#_step-2-convert-declared-element-to-target-uris">11.6.2.2. Step 2: Convert Declared Element to Target URIs</a></li>
<li><a href="#_step-3-filter-potential-resources">11.6.2.3. Step 3: Filter Potential Resources</a></li>
<li><a href="#_step-4-search-references-in-resource">11.6.2.4. Step 4: Search References in Resource</a></li>
<li><a href="#_limitations-and-possible-enhancements">11.6.2.5. Limitations and Possible Enhancements</a></li>
</ul>
</li>
</ul>
</li>
</ul>
</li>
<li><a href="#_compilation">12. Compilation</a>
<ul class="sectlevel2">
<li><a href="#chap:compilation">12.1. Introduction</a>
<ul class="sectlevel3">
<li><a href="#sec:general_design_rationals">12.1.1. General design rationals</a>
<ul class="sectlevel4">
<li><a href="#sec:logging_and_error_reporting">12.1.1.1. Logging and error reporting</a></li>
<li><a href="#sec:progress_monitor">12.1.1.2. Progress monitor</a></li>
</ul>
</li>
<li><a href="#sec:Xtext_Integration">12.1.2. Xtext Integration</a>
<ul class="sectlevel4">
<li><a href="#sec:xtext_default_behaviour">12.1.2.1. Xtext default behaviour</a></li>
<li><a href="#sec:n4js_requirements">12.1.2.2. N4JS requirements</a></li>
<li><a href="#sec:compiler_discovery_in_ui">12.1.2.3. Compiler discovery in UI</a></li>
<li><a href="#sec:compiler_discovery_in_headless">12.1.2.4. Compiler discovery in headless</a></li>
<li><a href="#sec:general_generator_implementation">12.1.2.5. General generator implementation</a></li>
<li><a href="#sec:general_generator_activation">12.1.2.6. General generator activation</a></li>
</ul>
</li>
<li><a href="#sec:Overview_of_Input_Models">12.1.3. Overview of the Input Models</a></li>
</ul>
</li>
<li><a href="#sec:Core_Generator">12.2. Generators</a>
<ul class="sectlevel3">
<li><a href="#sec:Compiler_Components">12.2.1. Generator Components</a></li>
<li><a href="#sec:Generator_architecture">12.2.2. Generator architecture</a></li>
<li><a href="#sec:Unified_Compiler_Configuration">12.2.3. Unified Compiler Configuration</a></li>
</ul>
</li>
<li><a href="#sec:Transpilers">12.3. Transpilers</a>
<ul class="sectlevel3">
<li><a href="#sec:Phases">12.3.1. Overview</a></li>
<li><a href="#relation-between-ast-and-im">12.3.2. Relation between AST and IM</a></li>
<li><a href="#implementation-overview">12.3.3. Implementation Overview</a></li>
<li><a href="#sec:Guidelines_for_Implementing_Transformations">12.3.4. Guidelines for Implementing Transformations</a></li>
<li><a href="#symbol-table-in-the-im">12.3.5. Symbol Table in the IM</a></li>
</ul>
</li>
<li><a href="#sec:N4JS_to_EcmaScript_Transpiler">12.4. N4JS-to-EcmaScript Transpiler</a>
<ul class="sectlevel3">
<li><a href="#sec:Overview_of_Transformations">12.4.1. Overview of Transformations</a></li>
<li><a href="#sec:Transpiling_members">12.4.2. Transpiling members</a>
<ul class="sectlevel4">
<li><a href="#sec:Transpiling_members__Delegating_members">12.4.2.1. Techniques for special member handling</a></li>
<li><a href="#sec:Transpiling_members__Partial_shadowing_of_getter_setter_pairs">12.4.2.2. Partial shadowing</a></li>
<li><a href="#sec:Transpiling_members__Consuming_or_inheriting_members_of_an_interface">12.4.2.3. Consuming or inheriting members of an interface</a></li>
<li><a href="#sec:Transpiling_members__Static_polyfill">12.4.2.4. Static polyfill</a></li>
<li><a href="#sec:Transpiling_members__API_implementation_stubs">12.4.2.5. API / implementation stubs</a></li>
</ul>
</li>
<li><a href="#sec:Support_for_incomplete_API_implementation_testing_in_the_N4JS_to_EcmaScript_5_Transpiler">12.4.3. Support for incomplete API implementation testing</a>
<ul class="sectlevel4">
<li><a href="#sec:Modifications_in_Impl_projects">12.4.3.1. Modifications in Impl projects</a></li>
<li><a href="#sec:Implementation_of_stub_generation">12.4.3.2. Implementation of stub-generation</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#sec:n4jsc_Headless_Compiler_Interface">12.5. n4jsc Headless Compiler Interface</a>
<ul class="sectlevel3">
<li><a href="#sec:building_the_headless_compiler">12.5.1. building the headless compiler</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#_execution">13. Execution</a>
<ul class="sectlevel2">
<li><a href="#sec:N4JS_Project_Execution_And_Linking_Model">13.1. N4JS Project Execution And Linking Model</a>
<ul class="sectlevel3">
<li><a href="#subsec:N4JS_Execution_With_NodeJS">13.1.1. N4JS Execution With NodeJS</a></li>
</ul>
</li>
<li><a href="#sec:N4JS_Execution_And_Linking_File">13.2. N4JS Execution And Linking File</a>
<ul class="sectlevel3">
<li><a href="#subsec:NodeJS_Specific_ELF">13.2.1. NodeJS Specific ELF</a></li>
</ul>
</li>
<li><a href="#sec:Runners-execution">13.3. Runners</a>
<ul class="sectlevel3">
<li><a href="#subsec:N4_Runtime_Environments_Convention">13.3.1. N4 Runtime Environments Convention</a></li>
<li><a href="#subsec:Passing_Information_from_IDE_to_Execution_Code_in_Runtime_Environment">13.3.2. Passing Information from IDE to Execution Code in Runtime Environment</a></li>
<li><a href="#subsec:Runners_Design">13.3.3. Runners Design</a></li>
</ul>
</li>
<li><a href="#sec:Legacy_Execution_Engine">13.4. Legacy Execution Engine</a></li>
<li><a href="#sec:Design">13.5. Design</a>
<ul class="sectlevel3">
<li><a href="#sec:Usage_Outside_N4JSIDE">13.5.1. Usage Outside N4JSIDE</a>
<ul class="sectlevel4">
<li><a href="#sec:Use_Node_with_Maven">13.5.1.1. Use Node with Maven</a></li>
</ul>
</li>
<li><a href="#sec:Usage_Inside_N4JSIDE">13.5.2. Usage Inside N4JSIDE</a></li>
</ul>
</li>
<li><a href="#sec:Runtime_Injection">13.6. Runtime Injection</a>
<ul class="sectlevel3">
<li><a href="#sec:Running_String_Code">13.6.1. Running String Code</a></li>
<li><a href="#sec:Running_File_Code">13.6.2. Running File Code</a></li>
<li><a href="#sec:Injection_Code_Example">13.6.3. Injection Code Example</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#_tests">14. Tests</a>
<ul class="sectlevel2">
<li><a href="#sec:Performance_Tests">14.1. Performance Tests</a>
<ul class="sectlevel3">
<li><a href="#sec:Synthetic_Performance_Tests">14.1.1. Synthetic Performance Tests</a>
<ul class="sectlevel4">
<li><a href="#sec:Design_of_Generator">14.1.1.1. Design of Generator</a></li>
<li><a href="#sec:Design_of_Performance_Test_Execution">14.1.1.2. Design of Performance Test Configuration and Execution</a></li>
<li><a href="#sec:JUnit_Configuration">14.1.1.3. JUnit Configuration</a></li>
<li><a href="#sec:JUnitBenchmark_Test_Configuration">14.1.1.4. JUnitBenchmark Test Configuration</a></li>
<li><a href="#sec:JUnitBenchmark_Report_Configuration">14.1.1.5. JUnitBenchmark Report Configuration</a></li>
<li><a href="#sec:JUnitBenchmark_Run_Configuration">14.1.1.6. JUnitBenchmark Run Configuration</a></li>
<li><a href="#sec:JUnitBenchmark_Example">14.1.1.7. JUnitBenchmark Example</a></li>
<li><a href="#sec:Note_on_Jenkins_Job">14.1.1.8. Note on Jenkins Job</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#sec:ECMA_Tests">14.2. ECMA Tests</a>
<ul class="sectlevel3">
<li><a href="#sec:Grammar_Tests">14.2.1. Grammar Tests</a>
<ul class="sectlevel4">
<li><a href="#sec:Negative_Tests">14.2.1.1. Negative Tests</a></li>
<li><a href="#sec:Test_Exclusion">14.2.1.2. Test Exclusion</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#sec:Integration_Tests">14.3. Integration Tests</a></li>
<li><a href="#sec:Test_Helpers">14.4. Test Helpers</a>
<ul class="sectlevel3">
<li><a href="#sec:Parameterized_N4JS_Tests">14.4.1. Parameterized N4JS tests</a>
<ul class="sectlevel4">
<li><a href="#sec:ParameterizedXtextRunner">14.4.1.1. ParameterizedXtextRunner</a></li>
<li><a href="#sec:TestCodeProvider">14.4.1.2. TestCodeProvider</a></li>
<li><a href="#sec:Example_Of_Parameterized_Parser_Test">14.4.1.3. Example of parameterized parser test</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#sec:Issue_Suppression">14.5. Issue Suppression</a></li>
<li><a href="#sec:Xpect_Tests">14.6. Xpect Tests</a>
<ul class="sectlevel3">
<li><a href="#sec:Xpect_Test_Setup">14.6.1. Xpect Test Setup</a></li>
<li><a href="#sec:Xpect_Issue_Suppression">14.6.2. Xpect Issue Suppression</a></li>
<li><a href="#sec:Xpect_Provided_Test_Methods">14.6.3. Xpect Provided Test Methods</a>
<ul class="sectlevel4">
<li><a href="#errors">14.6.3.1. errors</a></li>
<li><a href="#warnings">14.6.3.2. warnings</a></li>
</ul>
</li>
<li><a href="#sec:N4JS_Specific_Xpect_Test_Methods">14.6.4. N4JS Specific Xpect Test Methods</a>
<ul class="sectlevel4">
<li><a href="#sec:XPECT_noerrors">14.6.4.1. noerrors and nowarnings</a></li>
<li><a href="#sec:XPECT_scope">14.6.4.2. scope</a></li>
<li><a href="#sec:XPECT_scopeWithPosition">14.6.4.3. scopeWithPosition</a></li>
<li><a href="#sec:XPECT_scopeWithResource">14.6.4.4. scopeWithResource</a></li>
<li><a href="#sec:XPECT_binding">14.6.4.5. binding</a></li>
<li><a href="#sec:XPECT_linkedPathname">14.6.4.6. linkedPathname</a></li>
<li><a href="#sec:XPECT_type_of">14.6.4.7. type of</a></li>
<li><a href="#sec:XPECT_expectedType">14.6.4.8. expectedType</a></li>
<li><a href="#sec:XPECT_elementKeyword">14.6.4.9. elementKeyword</a></li>
<li><a href="#sec:XPECT_accessModifier">14.6.4.10. accessModifier</a></li>
<li><a href="#sec:XPECT_compileResult">14.6.4.11. compileResult</a></li>
<li><a href="#sec:XPECT_output">14.6.4.12. output</a></li>
<li><a href="#sec:XPECT_outputRegEx">14.6.4.13. outputRegEx</a></li>
<li><a href="#sec:XPECT_calculatedAccessModifier">14.6.4.14. calculatedAccessModifier</a></li>
<li><a href="#sec:XPECT_spec">14.6.4.15. spec</a></li>
<li><a href="#sec:XPECT_deadCode">14.6.4.16. deadCode</a></li>
<li><a href="#sec:XPECT_returnOrThrows">14.6.4.17. returnOrThrows</a></li>
<li><a href="#sec:XPECT_lint">14.6.4.18. lint</a></li>
<li><a href="#sec:XPECT_lintFails">14.6.4.19. lintFails</a></li>
</ul>
</li>
<li><a href="#sec:FIXME_Xpect_modifier">14.6.5. FIXME Xpect modifier</a></li>
<li><a href="#sec:Expectmatrix_Xpect_Test_Methods">14.6.6. Expectmatrix Xpect tests</a></li>
<li><a href="#xpect-lint-tests">14.6.7. Xpect Lint Tests</a></li>
</ul>
</li>
<li><a href="#xpect-proposal-tests">14.7. Xpect Proposal Tests</a>
<ul class="sectlevel3">
<li><a href="#sec:Validation_vs__Non_Validation">14.7.1. Validation vs. Non-Validation</a></li>
<li><a href="#sec:General_Proposal_Test_Features">14.7.2. General Proposal Test Features</a>
<ul class="sectlevel4">
<li><a href="#sec:Test_Variables">14.7.2.1. Test Variables</a></li>
<li><a href="#sec:Location_and_Selection">14.7.2.2. at – Location and Selection</a></li>
<li><a href="#sec:Multi_Line_Expectations_in_Proposal_Tests">14.7.2.3. Multi Line Expectations in Proposal Tests</a></li>
<li><a href="#sec:Timeout">14.7.2.4. Timeout and Performance</a></li>
</ul>
</li>
<li><a href="#proposals-verify-existence-of-proposals">14.7.3. proposals – Verify Existence of Proposals</a></li>
<li><a href="#sec:Verify_displayed_string">14.7.4. display – Verify displayed string</a></li>
<li><a href="#sec:Apply_Proposal">14.7.5. apply – Apply Proposal</a>
<ul class="sectlevel4">
<li><a href="#resource-application-in-other-files">14.7.5.1. resource – application in other files</a></li>
</ul>
</li>
<li><a href="#sec:Content_Assist_Cycling">14.7.6. kind – Content Assist Cycling</a></li>
<li><a href="#fileValidVerify-validation-status">14.7.7. fileValid – Verify validation status</a></li>
</ul>
</li>
<li><a href="#sec:Apply_Proposal_And_Execute_Tests">14.8. Apply Proposal And Execute Tests</a></li>
<li><a href="#sec:Organize_Imports_Test">14.9. Organize Imports Test</a>
<ul class="sectlevel3">
<li><a href="#organizeimports">14.9.1. organizeImports</a></li>
</ul>
</li>
<li><a href="#sec:Access_Control_Test">14.10. Access Control Test</a>
<ul class="sectlevel3">
<li><a href="#test-scenarios">14.10.1. Test Scenarios</a></li>
<li><a href="#n4js-code-generator">14.10.2. N4JS Code Generator</a></li>
<li><a href="#xtext-issue-matcher">14.10.3. Xtext Issue Matcher</a></li>
</ul>
</li>
<li><a href="#sec:Smoke_Tests">14.11. Smoke Tests</a>
<ul class="sectlevel3">
<li><a href="#how-to-handle-smoke-test-errors">14.11.1. How to handle smoke test errors?</a></li>
<li><a href="#smoketester-and-exceptionanalyzer">14.11.2. SmokeTester and ExceptionAnalyzer</a></li>
</ul>
</li>
<li><a href="#sec:UI_Tests_with_SWTBot">14.12. UI Tests with SWTBot</a>
<ul class="sectlevel3">
<li><a href="#writing-swtbot-tests">14.12.1. Writing SWTBot Tests</a></li>
<li><a href="#running-swtbot-tests">14.12.2. Running SWTBot Tests</a></li>
</ul>
</li>
<li><a href="#sec:Debugging_UI_Tests">14.13. Debugging UI Tests</a>
<ul class="sectlevel3">
<li><a href="#sec:Connecting_to_the_X_server_on_the_build_node">14.13.1. Connecting to the X-server on the build-node</a></li>
<li><a href="#sec:Tools_to_investigate_the_java_stack">14.13.2. Tools to investigate the java-stack</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#_ui-concepts">15. UI Concepts</a>
<ul class="sectlevel2">
<li><a href="#sec:User_Interface_Concepts">15.1. User Interface Concepts</a>
<ul class="sectlevel3">
<li><a href="#sec:Eclipse_UI_Concepts">15.1.1. Eclipse UI Concepts</a>
<ul class="sectlevel4">
<li><a href="#sec:Label_Provider">15.1.1.1. Label Provider</a></li>
<li><a href="#sec:Markers">15.1.1.2. Markers</a></li>
<li><a href="#sec:Commands__Toolbar_and_Menus">15.1.1.3. Commands, Toolbar and Menus</a></li>
<li><a href="#sec:Content_Assist">15.1.1.4. Content Assist</a></li>
<li><a href="#sec:Quick_Fixes">15.1.1.5. Quick Fixes</a></li>
<li><a href="#sec:Quick_Assist">15.1.1.6. Quick Assist</a></li>
<li><a href="#sec:Clean_Up_Actions">15.1.1.7. Clean Up Actions</a></li>
<li><a href="#sec:Save_Actions">15.1.1.8. Save Actions</a></li>
<li><a href="#sec:Auto_Edit">15.1.1.9. Auto Edit</a></li>
<li><a href="#sec:Template_Proposals">15.1.1.10. Template Proposals</a></li>
<li><a href="#sec:Outline_View___Quick_Outline">15.1.1.11. Outline View / Quick Outline</a></li>
<li><a href="#sec:Navigator__Package_Explorer__Project_Explorer">15.1.1.12. Navigator, Package Explorer, Project Explorer</a></li>
<li><a href="#sec:Hyperlinking_and_Navigation">15.1.1.13. Hyperlinking and Navigation</a></li>
<li><a href="#sec:Syntax_and_Semantic_Coloring">15.1.1.14. Syntax and Semantic Coloring</a></li>
<li><a href="#sec:Code_Formatter">15.1.1.15. Code Formatter</a></li>
<li><a href="#sec:Wizards">15.1.1.16. Wizards</a></li>
<li><a href="#sec:Cheat_Sheets">15.1.1.17. Cheat Sheets</a></li>
<li><a href="#sec:Context_sensitive_Help">15.1.1.18. Context-sensitive Help</a></li>
<li><a href="#sec:Hovers">15.1.1.19. Hovers</a></li>
<li><a href="#sec:Folding">15.1.1.20. Folding</a></li>
<li><a href="#sec:Customizable_validation___severity">15.1.1.21. Customizable validation / severity</a></li>
<li><a href="#sec:Proposals">15.1.1.22. Proposals</a></li>
</ul>
</li>
<li><a href="#sec:Non_Eclipse_UI_Concepts">15.1.2. Non-Eclipse UI Concepts</a>
<ul class="sectlevel4">
<li><a href="#sec:Overlays">15.1.2.1. Overlays</a></li>
<li><a href="#sec:Goto__Inferred__Type">15.1.2.2. Goto (Inferred) Type</a></li>
<li><a href="#sec:Postfix_Completion">15.1.2.3. Postfix Completion</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#_user-interface-resources">15.2. User Interface Resources</a>
<ul class="sectlevel3">
<li><a href="#_icons">15.2.1. Icons</a>
<ul class="sectlevel4">
<li><a href="#_eclipse-platform-icons">15.2.1.1. Eclipse Platform Icons</a></li>
<li><a href="#_n4js-specific-icons">15.2.1.2. N4JS Specific Icons</a></li>
<li><a href="#_high-resolution-icons">15.2.1.3. High Resolution Icons</a></li>
</ul>
</li>
</ul>
</li>
</ul>
</li>
<li><a href="#_formatting">16. Formatting</a>
<ul class="sectlevel2">
<li><a href="#sec:FmtObjective">16.1. Objective</a>
<ul class="sectlevel3">
<li><a href="#sec:FmtFormatting_Comments">16.1.1. Formatting Comments</a></li>
</ul>
</li>
<li><a href="#sec:FmtArchitecture">16.2. Architecture</a>
<ul class="sectlevel3">
<li><a href="#sec:Implementation_example">16.2.1. Implementation example</a></li>
</ul>
</li>
<li><a href="#sec:FmtFormatter_Implementation_Guidelines">16.3. Formatter Implementation Guidelines</a></li>
<li><a href="#sec:FmtConfiguration">16.4. Configuration</a></li>
<li><a href="#sec:FmtUI_Integration">16.5. UI Integration</a></li>
<li><a href="#sec:FmtUnit_Testing_with_Xpect">16.6. Unit Testing with Xpect</a></li>
</ul>
</li>
<li><a href="#_external-libraries">17. External Libraries</a>
<ul class="sectlevel2">
<li><a href="#sec:Major_Components">17.1. Major Components</a>
<ul class="sectlevel3">
<li><a href="#subsec:External_Resources">17.1.1. External Resources</a></li>
<li><a href="#subsec:External_Library_Workspace">17.1.2. External Library Workspace</a></li>
<li><a href="#subsec:External_Library_Preference_Store">17.1.3. External Library Preference Store</a></li>
<li><a href="#subsec:npm_Manager">17.1.4. Library Manager</a></li>
<li><a href="#subsec:External_Library_Builder_Helper">17.1.5. External Library Builder</a></li>
<li><a href="#subsec:External_Library_Xtext_Index_Persister">17.1.6. External Library Xtext Index Persister</a></li>
<li><a href="#subsec:External_Library_Preference_Page">17.1.7. External Library Preference Page</a></li>
</ul>
</li>
<li><a href="#sec:Headless_External_Library_Support">17.2. Headless External Library Support</a>
<ul class="sectlevel3">
<li><a href="#_custom-npm-settings">17.2.1. Custom npm settings</a></li>
</ul>
</li>
<li><a href="#sec:lmFutureWork">17.3. Future Work</a>
<ul class="sectlevel3">
<li><a href="#subsec:lmMultipleDependencyScope">17.3.1. Multiple Dependency Scope</a></li>
<li><a href="#subsec:lmRunTestsFromLibrary">17.3.2. Run Tests from TestLibrary</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#sec:JSON_Support">18. JSON Support</a>
<ul class="sectlevel2">
<li><a href="#sec:JSON_Parser">18.1. JSON Parser</a>
<ul class="sectlevel3">
<li><a href="#sec:JSON_Parser_Unicode_Escaping">18.1.1. Escaping Unicode Control Characters in String Literals</a></li>
<li><a href="#sec:JSON_Parser_Empty_Text">18.1.2. Empty Text</a></li>
<li><a href="#sec:JSON_Parser_Nested_Structures">18.1.3. Nested Structures</a></li>
<li><a href="#sec:JSON_Parser_Whitespace">18.1.4. Whitespace</a></li>
<li><a href="#sec:JSON_Parser_Comments">18.1.5. Comments</a></li>
</ul>
</li>
<li><a href="#sec:JSON_Language_Extensions">18.2. JSON Language Extensions</a>
<ul class="sectlevel3">
<li><a href="#sec:JSON_Validator_Extensions">18.2.1. JSON Validator Extensions</a>
<ul class="sectlevel4">
<li><a href="#sec:File_Specitic_Validator_Extensions">18.2.1.1. File-Specific Validator Extensions</a></li>
<li><a href="#sec:JSON_Declarative_JSON_Validator_Extensions">18.2.1.2. Declarative JSON Validator Extensions</a></li>
</ul>
</li>
<li><a href="#_json-resource-description-strategy">18.2.2. JSON Resource Description Strategy</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#_jsdoc">19. JSDoc</a>
<ul class="sectlevel2">
<li><a href="#sec:Design_Rationale">19.1. Design Rationale</a>
<ul class="sectlevel3">
<li><a href="#_general-design">19.1.1. General Design</a></li>
<li><a href="#sec:Type_Expressions">19.1.2. Type Expressions</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#_docexporter">20. DocExporter</a>
<ul class="sectlevel2">
<li><a href="#sec:Specification_Exporter">20.1. Specification Exporter</a></li>
</ul>
</li>
<li><a href="#_rename-refactoring">21. Rename Refactoring</a>
<ul class="sectlevel2">
<li><a href="#_rename-refactoring-ui-interaction">21.1. Rename Refactoring UI interaction</a></li>
<li><a href="#_renameelementprocessor-interaction">21.2. RenameElementProcessor interaction</a></li>
</ul>
</li>
<li><a href="#chap:flowgraphs">22. Flow Graphs</a>
<ul class="sectlevel2">
<li><a href="#sec:flowgraphs_overview">22.1. Flow graphs overview</a>
<ul class="sectlevel3">
<li><a href="#_internal-graph">22.1.1. Internal graph</a></li>
<li><a href="#_optimizations">22.1.2. Optimizations</a></li>
<li><a href="#_api-for-client-analyses">22.1.3. API for client analyses</a>
<ul class="sectlevel4">
<li><a href="#_mapping-from-internal-to-ast-elements">22.1.3.1. Mapping from internal to AST elements</a></li>
<li><a href="#_graph-visitor">22.1.3.2. Graph visitor</a></li>
<li><a href="#_graph-explorer">22.1.3.3. Graph explorer</a></li>
<li><a href="#_branch-walker">22.1.3.4. Branch walker</a></li>
<li><a href="#_example-1-compute-string-for-each-path">22.1.3.5. Example 1: Compute string for each path</a></li>
<li><a href="#_path-quantor">22.1.3.6. Path quantor</a></li>
</ul>
</li>
<li><a href="#_control-flow-analyses">22.1.4. Control flow analyses</a>
<ul class="sectlevel4">
<li><a href="#_dead-code-analysis">22.1.4.1. Dead code analysis</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#sec:dataflow">22.2. Dataflow</a>
<ul class="sectlevel3">
<li><a href="#_dataflow-graph">22.2.1. Dataflow graph</a></li>
<li><a href="#_dataflow-analyses">22.2.2. Dataflow analyses</a>
<ul class="sectlevel4">
<li><a href="#_def-def-def-nothing-analysis">22.2.2.1. Def&#8594;Def / Def&#8594;Nothing analysis</a></li>
<li><a href="#_def-use-decl-analysis">22.2.2.2. Def|Use&#8592;Decl analysis</a></li>
</ul>
</li>
</ul>
</li>
</ul>
</li>
<li><a href="#sec:publish-npms-to-public">23. Publish npms</a>
<ul class="sectlevel2">
<li><a href="#sec:publish-npms-n4js-maven">23.1. Publish n4js-libs to during maven build</a></li>
</ul>
</li>
<li><a href="#sec:Hints">Appendix A: Hints</a>
<ul class="sectlevel2">
<li><a href="#sec:XtextInjection">A.1. Xtext Injection</a>
<ul class="sectlevel3">
<li><a href="#sec:DI_MultipleInjectors_Singletons">A.1.1. Multiple Injectors and Singletons</a>
<ul class="sectlevel4">
<li><a href="#sec:DI_avoid_duplicate_singletons">A.1.1.1. Avoiding duplicate singletons</a>
<ul class="sectlevel5">
<li><a href="#sec:DI_binding_in_shared">A.1.1.1.1. Defining binding in the shared injector</a></li>
<li><a href="#sec:DI_binding_in_custom">A.1.1.1.2. Defining binding in the custom injector</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#sec:DI_Hints">A.1.2. Dependency Injection Hints</a>
<ul class="sectlevel4">
<li><a href="#sec:DI_custom_bundle">A.1.2.1. Use DI in custom bundle, use DI with extensions</a>
<ul class="sectlevel5">
<li><a href="#sec:DI_custom_bundle_problem">A.1.2.1.1. Problem</a></li>
<li><a href="#sec:DI_custom_bundle_solution">A.1.2.1.2. Solution</a></li>
</ul>
</li>
<li><a href="#sec:Access_Other_DSL_Injector">A.1.2.2. How do I get the Guice Injector of my language?</a>
<ul class="sectlevel5">
<li><a href="#sec:DSL_Injector_UI_context">A.1.2.2.1. UI context</a></li>
<li><a href="#sec:DSL_Injector_Non_UI_context">A.1.2.2.2. Non UI context but with injection context</a></li>
<li><a href="#sec:DSL_Injector_Non_UI_non_injection_context">A.1.2.2.3. Non UI context without injection context</a></li>
</ul>
</li>
<li><a href="#sec:Cancel_Indicator">A.1.2.3. How do I get cancel indicators in different contexts?</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#sec:Eclipse">A.2. Eclipse</a>
<ul class="sectlevel3">
<li><a href="#sec:Show_Xtext_Index">A.2.1. Show the current Xtext index</a></li>
<li><a href="#sec:Plugin_spy">A.2.2. Plug-in spy</a></li>
</ul>
</li>
<li><a href="#sec:Maven-hints">A.3. Maven</a>
<ul class="sectlevel3">
<li><a href="#how-to-check-for-maven-mojo-updates">A.3.1. How to check for Maven MOJO updates</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#_module-loading">Appendix B: Module Loading</a>
<ul class="sectlevel2">
<li><a href="#sec:Dependency_Management">B.1. Dependency Management</a></li>
<li><a href="#ecmascript-modules">B.2. ECMAScript Modules</a>
<ul class="sectlevel3">
<li><a href="#sec:ES5_Modules_Systems">B.2.1. ES5 Modules Systems</a></li>
<li><a href="#sec:ES6_Modules">B.2.2. ES6 Modules</a></li>
</ul>
</li>
<li><a href="#sec:ECMAScript_Module_Loaders">B.3. ECMAScript Module Loaders</a>
<ul class="sectlevel3">
<li><a href="#sec:ES6_Module_Loaders">B.3.1. ES6 Module Loaders</a></li>
<li><a href="#sec:Polyfills_for_ES6_Module_Loaders">B.3.2. Polyfills for ES6 Module Loaders</a>
<ul class="sectlevel4">
<li><a href="#sec:es6_module_loader">B.3.2.1. es6-module-loader</a></li>
<li><a href="#sec:SystemJS">B.3.2.2. SystemJS</a></li>
<li><a href="#sec:Demo">B.3.2.3. Demo</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#case-study-typescript">B.4. Case Study : TypeScript</a>
<ul class="sectlevel3">
<li><a href="#sec:ES6_Modules_Support">B.4.1. ES6 Modules Support</a></li>
<li><a href="#sec:TypeScript_and_Module_Loading">B.4.2. TypeScript and Module Loading</a></li>
</ul>
</li>
<li><a href="#sec:Cyclic_Dependencies">B.5. Cyclic Dependencies</a>
<ul class="sectlevel3">
<li><a href="#sec:Setup">B.5.1. Setup</a></li>
<li><a href="#sec:Transpile_and_Execute">B.5.2. Transpile and Execute</a>
<ul class="sectlevel4">
<li><a href="#sec:Module_Format___AMD">B.5.2.1. Module Format = AMD</a></li>
<li><a href="#sec:Module_Format___CommonJS">B.5.2.2. Module Format = CommonJS</a></li>
<li><a href="#sec:Module_Format___SystemJS">B.5.2.3. Module Format = SystemJS</a></li>
</ul>
</li>
<li><a href="#sec:Conclusion">B.5.3. Conclusion</a></li>
</ul>
</li>
<li><a href="#system.register-as-transpilation-target">B.6. System.register as transpilation target</a>
<ul class="sectlevel3">
<li><a href="#sec:Introduction">B.6.1. Introduction</a>
<ul class="sectlevel4">
<li><a href="#sec:External_Transpilers">B.6.1.1. External Transpilers</a></li>
<li><a href="#sec:Example_of_a_System_register_module">B.6.1.2. Example of a System.register module</a></li>
</ul>
</li>
<li><a href="#sec:Structure_of_a_System_register_module">B.6.2. Structure of a System.register module</a></li>
<li><a href="#_transpilation-hints">B.6.3. Transpilation Hints</a>
<ul class="sectlevel4">
<li><a href="#sec:Handling_Imports">B.6.3.1. Handling Imports</a></li>
<li><a href="#sec:__exportFn__">B.6.3.2. &lt;&lt;exportFn&gt;&gt;</a></li>
<li><a href="#sec:Handling_Exports">B.6.3.3. Handling Exports</a></li>
</ul>
</li>
<li><a href="#sec:Examples_w__Circular_Dependencies">B.6.4. Examples w/ Circular Dependencies</a></li>
<li><a href="#sec:N4JS_Examples_w__Circular_Dependencies">B.6.5. N4JS Examples w/ Circular Dependencies</a>
<ul class="sectlevel4">
<li><a href="#sec:Unresolved_Cyclic_Dependencies">B.6.5.1. Unresolved Cyclic Dependencies</a></li>
<li><a href="#sec:Variables___Functions">B.6.5.2. Examples with Variables &amp; Functions</a></li>
<li><a href="#sec:Classes">B.6.5.3. Examples with Classes</a></li>
<li><a href="#sec:Examples_with_SubClassing">B.6.5.4. Examples with SubClassing</a></li>
<li><a href="#sec:Miscellaneous">B.6.5.5. Miscellaneous</a></li>
</ul>
</li>
<li><a href="#_resources">B.6.6. Resources</a></li>
</ul>
</li>
<li><a href="#sec:CommonJS_as_transpilation_target">B.7. CommonJS as transpilation target</a>
<ul class="sectlevel3">
<li><a href="#_introduction-2">B.7.1. Introduction</a></li>
<li><a href="#sec:Transpilation_Hints">B.7.2. Transpilation Hints</a>
<ul class="sectlevel4">
<li><a href="#sec:Import_Statements">B.7.2.1. Import Statements</a></li>
<li><a href="#sec:Export_Statements">B.7.2.2. Export Statements</a></li>
<li><a href="#sec:Tracking_Live_Bindings">B.7.2.3. Tracking Live Bindings</a></li>
<li><a href="#sec:A_complete_example">B.7.2.4. A complete example</a></li>
</ul>
</li>
<li><a href="#_resources-2">B.7.3. Resources</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#sec:License">Appendix C: License</a></li>
<li><a href="#sec:Acronyms">Appendix D: Acronyms</a></li>
<li><a href="#_bibliography-and-footnotes">Appendix E: Bibliography and Footnotes</a></li>
</ul>
</div>
</div>
<div id="content">
<div id="preamble">
<div class="sectionbody">
<div class="paragraph center">
<p><strong>Last Updated: 2019-08-08</strong></p>
</div>
<div class="paragraph center">
<p><strong>Authors:</strong><br>
Jens von Pilgrim, Jakub Siberski, Mark-Oliver Reiser, Torsten Krämer, Ákos Kitta, Sebastian Zarnekow, Lorenzo Bettini, Jörg Reichert, Kristian Duske, Marcus Mews, Minh Quang Tran, Luca Beurer-Kellner</p>
</div>
<div style="page-break-after: always;"></div>
<div class="paragraph">
<p>This document contains the N4JS Design and Implementation documentation.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_introduction"><a class="anchor" href="#_introduction"></a><a class="link" href="#_introduction">1. Introduction</a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>This document describes design aspects of the N4JS compiler and IDE. It relies on the following N4JS related specifications:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>N4JS Language Specification [<a href="#N4JSSpec">N4JSSpec</a>]</p>
</li>
</ul>
</div>
<div class="sect2">
<h3 id="notation"><a class="anchor" href="#notation"></a><a class="link" href="#notation">1.1. Notation</a></h3>
<div class="paragraph">
<p>We reuse the notation specified in [<a href="#N4JSSpec">N4JSSpec</a>].</p>
</div>
</div>
<div class="sect2">
<h3 id="sec:IDE_Overview"><a class="anchor" href="#sec:IDE_Overview"></a><a class="link" href="#sec:IDE_Overview">1.2. IDE Components</a></h3>
<div class="paragraph">
<p>The N4JS and N4JSIDE components are organized via features. The following features with included plugins are defined
(the common prefix "org.eclipse.n4js" is omitted at the plugin name):</p>
</div>
<table class="tableblock frame-all grid-all spread">
<colgroup>
<col style="width: 5.8823%;">
<col style="width: 11.7647%;">
<col style="width: 82.353%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Feature</th>
<th class="tableblock halign-left valign-top">Plugin</th>
<th class="tableblock halign-left valign-top">Description</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock"><strong>org.eclipse.n4js.lang.sdk</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">N4JS core language with parser, validation etc.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">org.eclipse.n4js</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Xtext grammar with generator and custom code for N4JS, scoping (and binding) implementation, basic validation (and Xsemantics type system).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">doc</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">(in doc folder) General documentation (including web page) written in AsciiDoc</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">external.libraries</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Support for N4JS libraries shipped with the IDE, i.e. core N4JS library and mangelhaft.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">ui</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">UI components for N4JS, e.g., proposal provider, labels, outline, quickfixes.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">jsdoc</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Parser and model for JSDoc</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">external.libraries.update</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Not included in feature</strong>. Updates the external library plugin</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock"><strong>org.eclipse.n4js.ts.sdk</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Type System</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">ts</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Xtext grammar with generator and custom code for type expressions and standalone type definitions.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">ts.model</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Xcore based types model with helper classes etc.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">ts.ui</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Xtext generated UI for type system, not really used as this TS files are not editable by users.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock"><strong>org.eclipse.n4js.unicode.sdk</strong></p></td>
<td class="tableblock halign-left valign-top"></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">common.unicode</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Xtext grammar with generator and custom code used by all other grammars for proper unicode support.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock"><strong>org.eclipse.n4js.regex.sdk</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Regular expression grammar and UI, used by N4JS grammar and UI</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">regex</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Xtext grammar with generator and custom code used by N4JS grammars for regular expressions.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">regex.ui</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">UI components for regular expressions, e.g., proposal provider, labels, outline, quickfixes.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock"><strong>org.eclipse.n4js.sdk</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This feature defines the N4JSIDE. It contains core UI plugins and all includes (almost all) other features!</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">environments</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Utility plugin, registers n4scheme for EMF proxy resolution.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">model</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Xcore based N4JS model with helper classes etc.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">product</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">N4JSIDE main application.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">releng.utils</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">(in releng folder) Contains utility classes only used for building the system, e.g., tools for generating antlr based parser with extended features.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">utils</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">general utilities</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">utils.ui</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">general UI utilities</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock"><strong>org.eclipse.n4js.compiler.sdk</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Compilers and Transpilers</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">generator.common</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Not included in feature, logically associated.</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">generator.headless</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">N4JS headless generator (i.e. command line compiler).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">transpiler</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Generic transpiler infrastructure</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">transpiler.es</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Transpiler to compile to EcmaScript</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock"><strong>org.eclipse.n4js.json.sdk</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">N4JS JSON</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">json</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Xtext grammar with generator and custom code for a extensible JSON language support. Used in N4JS for the project description in terms of a <code>package.json</code> file.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">json.ui</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">UI components for extensible JSON language support, e.g., proposal provider, labels, outline.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">json.model</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Not included in feature, logically associated.</strong> Xcore based model for the JSON language.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock"><strong>org.eclipse.n4js.semver.sdk</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Semantic version string support.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">semver</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Parser and tools for semantic version strings.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">semver.ui</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">UI tools for semantic version strings.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">semver.model</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Not included in feature, logically associated.</strong> Xcore model of semantic version strings.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock"><strong>org.eclipse.n4js.runner.sdk</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Runners for executing N4JS or JavaScript code</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">runner</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Generic interfaces and helper for runners, i.e. JavaScript engines executing N4JS or JavaScript code.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">runner.chrome</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Runner for executing N4JS or JavaScript with Chrome.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">runner.chrome.ui</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">UI classes for launching the Chrome runner via the org.eclipse.debug.ui</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">runner.nodejs</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Runner for executing N4JS or JavaScript with node.js.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">runner.nodejs.ui</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">UI classes for launching the node.js runner via the org.eclipse.debug.ui</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">runner.ui</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Generic interfaces for configuring N4JS runner via the debug ui.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock"><strong>org.eclipse.n4js.tester.sdk</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Runners and UI for tests (via mangelhaft).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">tester</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Generic interfaces and helper for testers, i.e. JavaScript engines executing N4JS tests (using mangelhaft).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">tester.nodejs</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Tester based on the nodejs runner for executing mangelhaft tests with node.js</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">tester.nodejs.ui</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">UI for showing test results.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">tester.ui</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Configuration of tests via the debug UI.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock"><strong>org.eclipse.n4js.jsdoc2spec.sdk</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">JSDoc 2 Specification</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">jsdoc2spec</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Exporter to generate API documentation with specification tests awareness</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">jsdoc2spec.ui</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">UI for API doc exporter</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock"><strong>org.eclipse.n4js.xpect.sdk</strong></p></td>
<td class="tableblock halign-left valign-top"></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">xpect</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Xpect test methods.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">xpect.ui</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">UI for running Xpext tests methods from the N4JSIDE (for creating bug reports).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock"><strong>org.eclipse.n4js.smith.sdk</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Feature for internal N4JS IDE plugins only intended for development (for example, the AST Graph view).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">smith</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Non-UI classes for tools for smiths, that is, tools for developers of the N4JS IDE such as AST views etc.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">smith.ui</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">UI classes for tools for smiths, that is, tools for developers of the N4JS IDE such as AST views etc.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock"><strong>org.eclipse.n4js.tests.helper.sdk</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Test helpers.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock"><strong>org.eclipse.n4js.dependencies.sdk</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Collection of all external non-ui dependencies, used for local mirroring of update sites.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock"><strong>org.eclipse.n4js.dependencies.ui.sdk</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Collection of all external ui dependencies, used for local mirroring of update sites.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="3"><p class="tableblock"><strong>uncategorized plugins</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">flowgraphs</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Control and data flow graph model and computer.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" colspan="2"><p class="tableblock"><strong>Fragments</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">not associated to features, only listed here for completeness</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">utils.logging</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Fragment only, configuration for loggers, in particular for the product and for the tests</p></td>
</tr>
</tbody>
</table>
<div class="sect3">
<h4 id="sec:Naming_Conventions"><a class="anchor" href="#sec:Naming_Conventions"></a><a class="link" href="#sec:Naming_Conventions">1.2.1. Naming Conventions</a></h4>
<div class="paragraph">
<p>In the above sections, tests were omitted. We use the following naming conventions (by example) for test and tests helper:</p>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
project
</td>
<td class="hdlist2">
<p>-</p>
</td>
</tr>
<tr>
<td class="hdlist1">
project.tests
</td>
<td class="hdlist2">
<p>tests for project, is a fragment</p>
</td>
</tr>
<tr>
<td class="hdlist1">
project.tests.helper
</td>
<td class="hdlist2">
<p>helper classes used ONLY by tests</p>
</td>
</tr>
<tr>
<td class="hdlist1">
project.tests.performance
</td>
<td class="hdlist2">
<p>performance tests</p>
</td>
</tr>
<tr>
<td class="hdlist1">
project.tests.integration
</td>
<td class="hdlist2">
<p>integration tests</p>
</td>
</tr>
<tr>
<td class="hdlist1">
project.ui
</td>
<td class="hdlist2">
<p>-</p>
</td>
</tr>
<tr>
<td class="hdlist1">
project.ui.tests
</td>
<td class="hdlist2">
<p>tests for ui project, fragment of project.ui</p>
</td>
</tr>
<tr>
<td class="hdlist1">
project.ui.tests.helper
</td>
<td class="hdlist2">
<p>helper classes used ONLY by tests</p>
</td>
</tr>
<tr>
<td class="hdlist1">
project.ui.tests.performance
</td>
<td class="hdlist2">
<p>-</p>
</td>
</tr>
<tr>
<td class="hdlist1">
tests.helper
</td>
<td class="hdlist2">
<p>general test helper</p>
</td>
</tr>
<tr>
<td class="hdlist1">
ui.tests.helper
</td>
<td class="hdlist2">
<p>general ui test helper</p>
</td>
</tr>
<tr>
<td class="hdlist1">
project.xpect.tests
</td>
<td class="hdlist2">
<p>xpect tests for the project, despite dependnecies to UI the can be executed as plain JUnit tests</p>
</td>
</tr>
<tr>
<td class="hdlist1">
project.xpect.ui.tests
</td>
<td class="hdlist2">
<p>xpect tests for the project, need to be executed as eclipse plugin tests</p>
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>Due to Maven, tests are in subfolder tests (incl. helpers), implementation bundles in plugins, and release engineering related bundles in releng.</p>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_eclipse-setup"><a class="anchor" href="#_eclipse-setup"></a><a class="link" href="#_eclipse-setup">2. Eclipse Setup</a></h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="_system-requirements"><a class="anchor" href="#_system-requirements"></a><a class="link" href="#_system-requirements">2.1. System Requirements</a></h3>
<div class="paragraph">
<p>In all cases, <a href="https://adoptopenjdk.net/">Java 11</a> is required to be installed on your system. <a href="https://nodejs.org/en/download/">Node.js</a> version 10+ is also required, and for some tests you need <a href="https://yarnpkg.com">Yarn</a> to be globally installed.</p>
</div>
</div>
<div class="sect2">
<h3 id="_contribute"><a class="anchor" href="#_contribute"></a><a class="link" href="#_contribute">2.2. Contribute</a></h3>
<div class="paragraph">
<p>Eclipse developers who want to develop N4JS itself should use the <a href="https://www.eclipse.org/downloads/">Oomph Eclipse installer</a>. The N4JS project is listed under "Eclipse Projects/N4JS"
This setup installs the correct Eclipse version, creates a new workspace and clones all projects into it (for details see below).</p>
</div>
<div class="sect3">
<h4 id="_eclipse-installer"><a class="anchor" href="#_eclipse-installer"></a><a class="link" href="#_eclipse-installer">2.2.1. Eclipse Installer</a></h4>
<div class="paragraph">
<p>The recommended way to install the Eclipse IDE and set up the workspace is to use the Eclipse Installer.
This installer is to be downloaded from <a href="https://wiki.eclipse.org/Eclipse_Installer" class="bare">https://wiki.eclipse.org/Eclipse_Installer</a></p>
</div>
<div class="paragraph">
<p>Run the installer and apply the following steps:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>change to "Advance Mode" via the menu (upper-right corner) (no need to move the installer)</p>
</li>
<li>
<p>select a product, e.g. "Eclipse IDE for Eclipse Committers" with product version "2019-06". Hint: Do not select "latest" because this will cause automatic updates which may lead to weird errors later on.</p>
</li>
<li>
<p>double-click the entry <strong>Eclipse Projects/N4JS</strong> so that it is shown in the catalog view below</p>
</li>
<li>
<p>on the next page, configure paths accordingly. You only have to configure the installation and workspace folder. You may want to use git with https instead of ssh.</p>
</li>
<li>
<p>start installation</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>The installer will then guide you through the rest of the installation. All plug-ins are downloaded and configured automatically, so is the workspace including downloading the git repository and setting up the workspace.</p>
</div>
<div class="paragraph">
<p>The workspace is configured automatically. This includes fetching the necessary git repository. If you have selected git with SSH you may run into problems. In this case you can re-run the scripts and select HTTPS instead, this should work in any case.</p>
</div>
<div class="paragraph">
<p>Eventually the installer scripts are done, that means the git repository has been cloned and the workspace has been configured (including the project set setup).
Now the automatic build kicks in as you can see in the status bar.</p>
</div>
<div class="paragraph">
<p>The build will show a lot of errors while still working. Eventually the whole project should have been compiled without any errors. Unfortunately, due to a <a href="https://github.com/eclipse/n4js/issues/1373">known issue</a>, two problems exists. Please have a look at the linked issue on how to fix that (it is quite easy).</p>
</div>
<div class="sect4">
<h5 id="_changing-the-setup-script"><a class="anchor" href="#_changing-the-setup-script"></a><a class="link" href="#_changing-the-setup-script">2.2.1.1. Changing the Setup Script</a></h5>
<div class="paragraph">
<p>The setup scripts is stored at</p>
</div>
<div class="paragraph">
<p><code>n4js/releng/org.eclipse.n4js.targetplatform/N4JS.setup</code></p>
</div>
<div class="paragraph">
<p>Details about Oomph-Setup scripts can be found at</p>
</div>
<div class="paragraph">
<p><a href="https://wiki.eclipse.org/Eclipse_Installer" class="bare">https://wiki.eclipse.org/Eclipse_Installer</a></p>
</div>
</div>
</div>
<div class="sect3">
<h4 id="_manual-ide-configuration"><a class="anchor" href="#_manual-ide-configuration"></a><a class="link" href="#_manual-ide-configuration">2.2.2. Manual IDE Configuration</a></h4>
<div class="admonitionblock warning">
<table>
<tr>
<td class="icon">
<i class="fa icon-warning" title=""></i>
</td>
<td class="content">
Manual IDE configuration is not recommended!
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>For a manual install, clone the code and import all top-level projects from the docs, features, plugins, releng, testhelpers, and tests folders. Activate the targetplatform contained in the <code>releng/org.eclipse.n4js.targetplatform/</code> project.</p>
</div>
<div class="paragraph">
<p>The N4JS IDE is developed with Eclipse 2019-06 or better since the system is based on Eclipse anyway.
It is almost impossible to use another IDE to develop Eclipse plugins. The list of required plugins includes:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Xtext/Xtend 2.18.0</p>
</li>
<li>
<p>Xcore 1.9.0</p>
</li>
<li>
<p>Xpect 0.2.0.201906240918 from <a href="https://ci.eclipse.org/xpect/job/Xpect-Integration-Release/20/artifact/org.eclipse.xpect.releng/p2-repository/target/repository/" class="bare">https://ci.eclipse.org/xpect/job/Xpect-Integration-Release/20/artifact/org.eclipse.xpect.releng/p2-repository/target/repository/</a></p>
</li>
</ul>
</div>
<div class="paragraph">
<p>It is important to use the latest version of Xtext and the corresponding service release of Xcore. You will find the latest version numbers and plugins used in the target platform definition at
<a href="https://github.com/eclipse/n4js/blob/master/releng/org.eclipse.n4js.targetplatform/org.eclipse.n4js.targetplatform.target" class="bare">https://github.com/eclipse/n4js/blob/master/releng/org.eclipse.n4js.targetplatform/org.eclipse.n4js.targetplatform.target</a></p>
</div>
<div class="paragraph">
<p>You may need to adjust some settings in Eclipse, most importantly</p>
</div>
<div class="ulist">
<ul>
<li>
<p><strong>Text file encoding</strong> to <code>Other: UTF-8</code> and</p>
</li>
<li>
<p><strong>New text file line delimiter</strong> to <code>Unix</code> .</p>
</li>
</ul>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_release-engineering"><a class="anchor" href="#_release-engineering"></a><a class="link" href="#_release-engineering">3. Release Engineering</a></h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="_nightly-build-on-eclipse-infrastructure"><a class="anchor" href="#_nightly-build-on-eclipse-infrastructure"></a><a class="link" href="#_nightly-build-on-eclipse-infrastructure">3.1. Nightly build on Eclipse infrastructure</a></h3>
<div class="paragraph">
<p>The N4JS IDE, headless n4jsc.jar, and the N4JS update site is being built on the Eclipse Common Build
Infrastructure (CBI). For this purpose the N4JS project is using a dedicated Jenkins instance, referred
to as a "Jenkins Instance Per Project" (JIPP) in Eclipse CBI documentation. At this time, the N4JS
project&#8217;s JIPP is running on the "old" infrastructure, not yet using docker. This will be migrated
at a later point in time.</p>
</div>
<div class="paragraph">
<p>The N4JS JIPP is available at: <a href="https://ci.eclipse.org/n4js/" class="bare">https://ci.eclipse.org/n4js/</a></p>
</div>
<div class="paragraph">
<p>The nightly build performs the following main steps:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>compile the N4JS implementation,</p>
</li>
<li>
<p>build the n4jsc.jar, the IDE products for MacOS, Windows, Linux, and the update site,</p>
</li>
<li>
<p>run tests,</p>
</li>
<li>
<p>sign the IDE product for macOS and package it in a .dmg file,</p>
</li>
<li>
<p>deploy to n4jsc.jar, IDE products and update sites to Eclipse download server (i.e. download.eclipse.org),</p>
</li>
<li>
<p>move all artifacts older than 7 days from download.eclipse.org to archive.eclipse.org.</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>Details about all the above steps can be found in the Jenkinsfile <code>eclipse-nightly.jenkinsfile</code>, located in
the root folder of the N4JS source repository on GitHub.</p>
</div>
<div class="paragraph">
<p>The most accurate documentation for our JIPP can be found at <a href="https://wiki.eclipse.org/IT_Infrastructure_Doc" class="bare">https://wiki.eclipse.org/IT_Infrastructure_Doc</a>.
Note that many other documents do not apply to our JIPP, at the moment, as they refer to the new
infrastructure, e.g. <a href="https://wiki.eclipse.org/CBI" class="bare">https://wiki.eclipse.org/CBI</a> and <a href="https://wiki.eclipse.org/Jenkins" class="bare">https://wiki.eclipse.org/Jenkins</a>.</p>
</div>
</div>
<div class="sect2">
<h3 id="_build-the-n4js-ide-from-command-line"><a class="anchor" href="#_build-the-n4js-ide-from-command-line"></a><a class="link" href="#_build-the-n4js-ide-from-command-line">3.2. Build the N4JS IDE from command line</a></h3>
<div class="paragraph">
<p>Ensure you have</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Java 11</p>
</li>
<li>
<p>Maven 3.2.x and</p>
</li>
<li>
<p>Node.js 8</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>installed on your system.</p>
</div>
<div class="paragraph">
<p>Clone the repository</p>
</div>
<div class="listingblock">
<div class="content">
<pre>git clone https://github.com/Eclipse/n4js.git</pre>
</div>
</div>
<div class="paragraph">
<p>Change to the n4js folder:</p>
</div>
<div class="listingblock">
<div class="content">
<pre>cd n4js</pre>
</div>
</div>
<div class="paragraph">
<p>Run the Maven build:</p>
</div>
<div class="listingblock">
<div class="content">
<pre>mvn clean verify</pre>
</div>
</div>
<div class="paragraph">
<p>You may have to increase the memory for maven via <code>export MAVEN_OPTS="-Xmx2048m"</code> (Unix) or <code>set MAVEN_OPTS="-Xmx2048m"</code> (Windows).</p>
</div>
<div class="paragraph">
<p>Available optional maven profiles are:</p>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
buildProduct
</td>
<td class="hdlist2">
<p>create IDE products (Windows, macOS, Linux) and a jar for headless compilation</p>
</td>
</tr>
<tr>
<td class="hdlist1">
execute-plugin-tests
</td>
<td class="hdlist2">
<p>run OSGi tests (without UI)</p>
</td>
</tr>
<tr>
<td class="hdlist1">
execute-plugin-ui-tests
</td>
<td class="hdlist2">
<p>run UI-based OSGi tests</p>
</td>
</tr>
<tr>
<td class="hdlist1">
execute-ecmas-tests
</td>
<td class="hdlist2">
<p>run ECMA test suite</p>
</td>
</tr>
<tr>
<td class="hdlist1">
execute-smoke-tests
</td>
<td class="hdlist2">
<p>run generated tests using corrupted source code as input</p>
</td>
</tr>
<tr>
<td class="hdlist1">
execute-accesscontrol-tests
</td>
<td class="hdlist2">
<p>run generated tests for checking accessibility of class/interface members</p>
</td>
</tr>
<tr>
<td class="hdlist1">
execute-hlc-integration-tests
</td>
<td class="hdlist2">
<p>run integration tests using the headless jar (requires docker!)</p>
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>Available system properties:</p>
</div>
<div class="hdlist">
<table>
<tr>
<td class="hdlist1">
noTests
</td>
<td class="hdlist2">
<p>suppress execution of all tests</p>
</td>
</tr>
<tr>
<td class="hdlist1">
startAndKeepVerdaccio
</td>
<td class="hdlist2">
<p>enforce starting and suppress stopping of the test verdaccio (see <a href="#sec:test-verdaccio">Test Verdaccio containing n4js-libs</a>)</p>
</td>
</tr>
</table>
</div>
<div class="sect3">
<h4 id="_publish-maven-tooling-code-org-eclipse-n4js-releng-util-code"><a class="anchor" href="#_publish-maven-tooling-code-org-eclipse-n4js-releng-util-code"></a><a class="link" href="#_publish-maven-tooling-code-org-eclipse-n4js-releng-util-code">3.2.1. Publish maven-tooling <code>org.eclipse.n4js.releng.util</code></a></h4>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title=""></i>
</td>
<td class="content">
For extending the N4JS-language in a different project, the <code>org.eclipse.n4js.releng.util</code> module needs to be published as a maven-plugin. You can deploy this SNAPSHOT-artifact to a local folder by providing the <code>local-snapshot-deploy-folder</code>-property pointing to an absolute path in the local file system:
</td>
</tr>
</table>
</div>
<div class="listingblock">
<div class="content">
<pre>mvn clean deploy -Dlocal-snapshot-deploy-folder=/var/lib/my/folder/local-mvn-deploy-repository</pre>
</div>
</div>
<div class="paragraph">
<p>The existence of <code>local-snapshot-deploy-folder</code> will trigger a profile enabling the deploy-goal for the project <code>org.eclipse.n4js.releng.util</code></p>
</div>
</div>
<div class="sect3">
<h4 id="sec:test-verdaccio"><a class="anchor" href="#sec:test-verdaccio"></a><a class="link" href="#sec:test-verdaccio">3.2.2. Test Verdaccio containing n4js-libs</a></h4>
<div class="paragraph">
<p>If profile <code>execute-hlc-integration-tests</code> is active, a local verdaccio instance is started and populated with
freshly-compiled n4js-libs (the libraries located under top-level folder <code>/n4js-libs</code>) and is stopped before the
end of the build. The verdaccio instance is started as a docker container called <code>n4js-test-verdaccio</code>.</p>
</div>
<div class="paragraph">
<p>When giving <code>-DstartAndKeepVerdaccio</code> on the command line, such a test verdaccio will always be started/populated but
never stopped, regardless of whether profile <code>execute-hlc-integration-tests</code> is active or not. This is useful to enforce
starting of the test verdaccio (even without running integration tests) and then reusing it in subsequent builds.</p>
</div>
</div>
<div class="sect3">
<h4 id="_generation-of-eclipse-help-for-spec-and-design-document"><a class="anchor" href="#_generation-of-eclipse-help-for-spec-and-design-document"></a><a class="link" href="#_generation-of-eclipse-help-for-spec-and-design-document">3.2.3. Generation of Eclipse help for spec and design document</a></h4>
<div class="paragraph">
<p>The HTML pages for N4JSSpec and N4JSDesign documents are generated from the Asciidoc sources in the project <code>org.eclipse.n4js.spec</code> <code>org.eclipse.n4js.design</code> by Asciispec. </p>
</div>
<div id="img:eclipse-help-doc-process" class="imageblock">
<div class="content">
<img src="chapters/03_releng/images/eclipse-help-process.svg" alt="Creating Eclipse help for N4JSSpec">
</div>
<div class="title">Figure 1. The process of creating Eclipse help for N4JSSpec</div>
</div>
<div class="paragraph">
<p>Figure <a href="#img:eclipse-help-doc-process">The process of creating Eclipse help for N4JSSpec</a> shows the generation process for N4JSSpec document. The process for N4JSDesign (and other adoc documents) is the same. The following explains the diagram.</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>Asciispec</code> is used to compile the source N4JSSpec Asciidoc into a single large <code>N4JSSpec.html</code> file which contains all the chapters. The use of the custom parameter <code>-a eclipse-help-mode</code> indicates that a special header and footer styles as well as CSS style should be used (i.e. no table of content menu, no download links etc.). Here, we are using the possibility provided by Asciidoctor to configure header/footer as well as CSS style via parameter <code>:docinfodir:</code> and <code>:stylesheet:</code>.</p>
</li>
<li>
<p>Our custom tool <code>Chunker</code> splits <code>N4JSSpec.html</code> (and other documents) into multiple chunked HTML files, each of which corresponds to either the <code>index</code> file or a chapter. It automatically re-writes internal links.</p>
</li>
<li>
<p>Another custom tool <code>EclipseHelpTOCGenerator</code> takes to Docbook file <code>N4JSSpec.xml</code> and generates an XML file describing the table of content (TOC) in the Eclipse format. This TOC file references the chunked HTML files above.</p>
</li>
<li>
<p>Another custom tool <code>IndexTocGenerator</code> takes to Docbook file <code>N4JSSpec.xml</code> similar to <code>EclipseHelpTOCGenerator</code>, but it generates an HTML fragment which can be embedded into the <code>index.html</code> page generated by the <code>Chunker</code> (Thus it has to run before the Chunker in that case).</p>
</li>
</ul>
</div>
</div>
</div>
<div class="sect2">
<h3 id="_updating-frameworks-and-dependencies"><a class="anchor" href="#_updating-frameworks-and-dependencies"></a><a class="link" href="#_updating-frameworks-and-dependencies">3.3. Updating frameworks and dependencies</a></h3>
<div class="sect3">
<h4 id="_update-of-eclipse-emf-xtext-etc"><a class="anchor" href="#_update-of-eclipse-emf-xtext-etc"></a><a class="link" href="#_update-of-eclipse-emf-xtext-etc">3.3.1. Update of Eclipse, EMF, Xtext, etc.</a></h4>
<div class="paragraph">
<p>For updating the N4JS IDE to a new version of Eclipse, EMF, Xtext, etc. follow these steps:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Create a new branch.</p>
</li>
<li>
<p>Bump versions of all dependencies mentioned in file <code>N4JS.setup</code>:</p>
<div class="olist loweralpha">
<ol class="loweralpha" type="a">
<li>
<p>Update all labels that refer to the version of the Ooomph setup (search for "label!" to find them).</p>
</li>
<li>
<p>Choose a new Eclipse version and define this in <code>N4JS.setup</code>.</p>
</li>
<li>
<p>For those other dependencies <em>that come with Eclipse</em> (e.g. EMF, Xtext) find out which version matches the chosen Eclipse version
and define that version in <code>N4JS.setup</code>.<br>
Tip: use the contents list of the SimRel you are targeting, e.g. <a href="https://projects.eclipse.org/releases/2019-03" class="bare">https://projects.eclipse.org/releases/2019-03</a></p>
</li>
<li>
<p>For those other dependencies <em>that are available via the Eclipse Orbit</em>, find out which version is the latest version available in
the Orbit and define that version in <code>N4JS.setup</code>.<br>
Tip: contents of the Eclipse Orbit can be found at <a href="https://download.eclipse.org/tools/orbit/downloads/" class="bare">https://download.eclipse.org/tools/orbit/downloads/</a><br>
(choose the correct link for the chosen Eclipse version!)</p>
</li>
<li>
<p>For all remaining dependencies (i.e. unrelated to Eclipse and not in Orbit), choose a version to use and define it in <code>N4JS.setup</code>.</p>
</li>
</ol>
</div>
</li>
<li>
<p>Check <code>Require-Bundle</code> sections of MANIFEST.MF files by searching for related bundle names or for <code>;bundle-version="</code>:</p>
<div class="olist loweralpha">
<ol class="loweralpha" type="a">
<li>
<p>There should be at most one version constraint for a specific bundle<br>
NOTE: the version constraints in the MANIFEST.MF files are just lower bounds and - at this time - we do not bump them to the latest version, in most cases.</p>
</li>
<li>
<p>There should be no version constraints to our bundles (i.e. <code>org.eclipse.n4js&#8230;&#8203;</code>)</p>
</li>
</ol>
</div>
</li>
<li>
<p>Review parent pom.xml files, i.e. <code>releng/org.eclipse.n4js.parent/pom.xml</code>:</p>
<div class="olist loweralpha">
<ol class="loweralpha" type="a">
<li>
<p>Update property <code>xtext-version</code>.</p>
</li>
<li>
<p>Check all other <code>*-version</code> properties and update them where needed.</p>
</li>
</ol>
</div>
</li>
<li>
<p>Update target platform file <code>org.eclipse.n4js.targetplatform.target</code> using Ooomph&#8217;s auto-generation:</p>
<div class="olist loweralpha">
<ol class="loweralpha" type="a">
<li>
<p>Start the Eclipse Installer.</p>
</li>
<li>
<p>Update the Eclipse Installer (using the button with the turning arrows).</p>
</li>
<li>
<p>On the second page, add the <code>N4JS.setup</code> file from your branch to the Eclipse Installer, using a GitHub raw(!) URL:<br>
<code><a href="https://raw.githubusercontent.com/eclipse/n4js/BRANCH_NAME/releng/org.eclipse.n4js.targetplatform/N4JS.setup" class="bare">https://raw.githubusercontent.com/eclipse/n4js/BRANCH_NAME/releng/org.eclipse.n4js.targetplatform/N4JS.setup</a></code></p>
</li>
<li>
<p>Ooomph a new development environment with this setup.</p>
</li>
<li>
<p>In the new Eclipse workspace created by Ooomph, the target platform file should have uncommitted changes:</p>
<div class="olist lowerroman">
<ol class="lowerroman" type="i">
<li>
<p>carefully review these changes, to be sure they make sense, and then</p>
</li>
<li>
<p>commit &amp; push those changes to your branch.</p>
</li>
</ol>
</div>
</li>
</ol>
</div>
</li>
<li>
<p>Thoroughly test the new versions, including some manual(!) tests:</p>
<div class="olist loweralpha">
<ol class="loweralpha" type="a">
<li>
<p>Run Jenkins builds.</p>
</li>
<li>
<p>Ooomph another N4JS development environment with Eclipse Installer.
This time, after Ooomphing is completed, the target platform file should no longer have any uncommitted changes.</p>
</li>
<li>
<p>Ensure the following types of tests can be executed locally in the newly installed Eclipse:</p>
<div class="olist lowerroman">
<ol class="lowerroman" type="i">
<li>
<p>plain JUnit tests (e.g. <code>org.eclipse.n4js.lang.tests</code>).</p>
</li>
<li>
<p>Plugin tests.</p>
</li>
<li>
<p>Plugin UI tests.</p>
</li>
<li>
<p>SWTBot tests.</p>
</li>
<li>
<p>Xpect tests (individual files and entire bundles; e.g. <code>org.eclipse.n4js.spec.tests</code>).</p>
</li>
<li>
<p>Xpect UI tests.</p>
</li>
</ol>
</div>
</li>
<li>
<p>Ensure an N4JS IDE product can be launched from within the newly installed Eclipse using the launch configuration
provided in the n4js repository.</p>
</li>
<li>
<p>After launching the N4JS IDE product, refresh the workspace and review/commit any changes in file <code>N4JS__IDE.launch</code>.</p>
</li>
<li>
<p>Download a product created in a Jenkins CI build and test it manually.</p>
</li>
<li>
<p>After merging to master: download a product created in a nightly build and test it manually.
Ensure signing and JRE bundling are still working properly.</p>
</li>
</ol>
</div>
</li>
</ol>
</div>
<div class="paragraph">
<p>All the above steps need to be performed in the <code>n4js-n4</code> repository, accordingly (e.g. file <code>N4JS-N4.setup</code>).</p>
</div>
</div>
<div class="sect3">
<h4 id="_update-of-the-embedded-jre"><a class="anchor" href="#_update-of-the-embedded-jre"></a><a class="link" href="#_update-of-the-embedded-jre">3.3.2. Update of the embedded JRE</a></h4>
<div class="paragraph">
<p>For updating the embedded JRE inside the N4JS IDE follow these steps:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Given a new JRE download location for Linux, MacOS and Windows with a common new version</p>
</li>
<li>
<p>Update the location related properties in the pom.xml files of</p>
<div class="olist loweralpha">
<ol class="loweralpha" type="a">
<li>
<p>n4js/builds/pom.xml</p>
</li>
<li>
<p>n4js/builds/org.eclipse.n4js.jre.linux.gtk.x86_64/pom.xml</p>
</li>
<li>
<p>n4js/builds/org.eclipse.n4js.jre.macosx.cocoa.x86_64/pom.xml</p>
</li>
<li>
<p>n4js/builds/org.eclipse.n4js.jre.win32.win32.x86_64/pom.xml</p>
</li>
</ol>
</div>
</li>
<li>
<p>Update the versions at all following locations:</p>
<div class="olist loweralpha">
<ol class="loweralpha" type="a">
<li>
<p>n4js/builds/org.eclipse.n4js.jre.linux.gtk.x86_64/META-INF/MANIFEST.MF</p>
</li>
<li>
<p>n4js/builds/org.eclipse.n4js.jre.linux.gtk.x86_64/META-INF/p2.inf</p>
</li>
<li>
<p>n4js/builds/org.eclipse.n4js.jre.macosx.cocoa.x86_64/META-INF/MANIFEST.MF</p>
</li>
<li>
<p>n4js/builds/org.eclipse.n4js.jre.macosx.cocoa.x86_64/META-INF/p2.inf</p>
</li>
<li>
<p>n4js/builds/org.eclipse.n4js.jre.win32.win32.x86_64/META-INF/MANIFEST.MF</p>
</li>
<li>
<p>n4js/builds/org.eclipse.n4js.jre.win32.win32.x86_64/META-INF/p2.inf</p>
</li>
</ol>
</div>
</li>
<li>
<p>Update the openjdk docker image used as base image in the "FROM" line at the top of all docker files:</p>
<div class="olist loweralpha">
<ol class="loweralpha" type="a">
<li>
<p>n4js-n4/jenkins/docker-build/Dockerfile</p>
</li>
</ol>
</div>
</li>
</ol>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_tips-and-tricks"><a class="anchor" href="#_tips-and-tricks"></a><a class="link" href="#_tips-and-tricks">4. Tips and Tricks</a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>In this chapter we collect some coding hints and guidelines on how to properly use the APIs of Eclipse, EMF, Xtext and
other dependencies we are using, as well as our own utilities and helpers.</p>
</div>
<div class="paragraph">
<p>This chapter is only about coding; add information on things like Eclipse setup or Maven/Jenkins to one of the preceding
chapters. Similarly, this chapter is intended to provide just a quick overview, check-list and reminder; add detailed
information and diagrams to one of the succeeding chapters.</p>
</div>
<div class="sect2">
<h3 id="_naming"><a class="anchor" href="#_naming"></a><a class="link" href="#_naming">4.1. Naming</a></h3>
<div class="ulist">
<ul>
<li>
<p>The internal handling of N4JS project names is non-trivial (due to the support for npm scopes), see
API documentation of <code>ProjectDescriptionUtils#isProjectNameWithScope(String)</code> for a detailed overview.
In short:</p>
<div class="ulist">
<ul>
<li>
<p><code>IN4JSProject#getProjectName()</code> and <code>IProject#getName()</code> return different values!</p>
</li>
<li>
<p>Avoid using the Eclipse project name, i.e. the return value of <code>IProject#getName()</code>, as far as possible
(only use it in UI code when actually dealing with what is shown in the Eclipse UI).</p>
</li>
<li>
<p>The last segment of an URI or path pointing to an N4JS project is <strong>not</strong> always the project name; use
utilities in <code>ProjectDescriptionUtils</code> instead, e.g. <code>#deriveN4JSProjectNameFromURI()</code>!
(However, given an URI or path pointing to a file inside an N4JS project, you can use its last segment
to obtain the file name.)</p>
</li>
</ul>
</div>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_logging"><a class="anchor" href="#_logging"></a><a class="link" href="#_logging">4.2. Logging</a></h3>
<div class="paragraph">
<p>In many situations developer needs to use some kind of logging. When in need, follow these rules:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Use <code>org.apache.log4j.Logger;</code> for logging. Other logging utilities (like java built in logger) are not configured.</p>
</li>
<li>
<p>do not use <code>System.out</code> nor <code>Sysetem.err</code> for logging. It is ok to use it for debugging purposes, but those calls
should never be merged to master. <em>(with exception of headless compiler, which uses them explicitly)</em></p>
</li>
<li>
<p>There is central logger configuration in <code>org.eclipse.n4js.utils.logging</code> (and <code>org.eclipse.n4js.utils.logging</code>) that should
be used</p>
<div class="olist loweralpha">
<ol class="loweralpha" type="a">
<li>
<p><code>log4j.xml</code> used for production</p>
</li>
<li>
<p><code>log4j_tests.xml</code> used when running tests</p>
</li>
</ol>
</div>
</li>
<li>
<p>in Eclipse run configurations logger has to be set properly, e.g.
<code>log4j.configuration=file:${workspace_loc:org.eclipse.n4js.utils.logging/log4j_tests.xml}</code></p>
</li>
<li>
<p>in maven configurations logger has to be set separately, e.g.
<code>-Dlog4j.configuration="file:${basedir}/../../plugins/org.eclipse.n4js.utils.logging/log4j_tests.xml</code></p>
</li>
</ol>
</div>
</div>
<div class="sect2">
<h3 id="_cancellation-handling"><a class="anchor" href="#_cancellation-handling"></a><a class="link" href="#_cancellation-handling">4.3. Cancellation Handling</a></h3>
<div class="paragraph">
<p>At various occasions, Xtext provides an instance of class <code>CancelIndicator</code> to allow our code to handle cancellation of
long-running task.</p>
</div>
<div class="paragraph">
<p>Some things to keep in mind:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>whenever a <code>CancelIndicator</code> is available any code that might not return immediately should implement proper
cancellation handling (as explained in the next items).</p>
</li>
<li>
<p>most importantly: reacting to a cancellation by returning early from a method is an anti-pattern that leads to
problems (client code might continue work on a canceled and thus invalid state); instead: throw an
<code>OperationCanceledException</code>!</p>
</li>
<li>
<p>don&#8217;t use <code>CancelIndicator#isCanceled()</code> for cancellation handling, except in certain special cases. A valid exception
case might be during logging to show a message like "operation was canceled".</p>
</li>
<li>
<p>instead, inject the Xtext service called <code>OperationCanceledManager</code> and invoke its method <code>#checkCanceled()</code>, passing-in
the cancel indicator (this method is null-safe; it will throw an <code>OperationCanceledException</code> in case a cancellation has
occurred). Don&#8217;t directly create and throw an <code>OperationCanceledException</code> yourself.</p>
</li>
<li>
<p>use the other methods provided by <code>OperationCanceledManager</code> when appropriate (see code of that class for details).</p>
</li>
<li>
<p>in try/catch blocks, when catching exceptions of a super type of <code>OperationCanceledException</code>, be sure to <strong>not suppress</strong>
cancellation exceptions. For example:</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-java" data-lang="java">// Java code
@Inject private OperationCanceledManager operationCanceledManager;
/** Returns true on success, false otherwise. */
public boolean doSomething(CancelIndicator ci) {
try {
// do something that might be canceled
return true;
} catch(Exception e) {
operationCanceledManager.propagateIfCancelException(e); // &lt;- IMPORTANT!
return false;
}
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>Try/finally blocks, on the other hand, do not need any special handling.</p>
</div>
</li>
<li>
<p>a cancel indicator can also be stored in the rule environment (see <code>RuleEnvironmentExtensions#addCancelIndicator()</code>). This
means:</p>
<div class="ulist">
<ul>
<li>
<p>if you create a rule environment completely from scratch and you have a cancel indicator at hand, add it to the rule
environment via <code>RuleEnvironmentExtensions#addCancelIndicator()</code> (not required when using <code>RuleEnvironmentExtensions#wrap()</code> for
deriving a rule environment from an existing one).</p>
</li>
<li>
<p>if you have a rule environment available, be sure to use its cancel indicator in long-running operations, i.e. with
code like:</p>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-java" data-lang="java">// Xtend code
import static extension org.eclipse.n4js.typesystem.utils.RuleEnvironmentExtensions.*
class C {
@Inject private OperationCanceledManager operationCanceledManager;
def void doSomething() {
for(a : aLotOfStuff) {
operationCanceledManager.checkCanceled(G.cancelIndicator);
// main work ...
}
}</code></pre>
</div>
</div>
</li>
</ul>
</div>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_caching"><a class="anchor" href="#_caching"></a><a class="link" href="#_caching">4.4. Caching</a></h3>
<div class="ulist">
<ul>
<li>
<p>Caching of external libraries (implemented in ExternalProjectMappings)</p>
<div class="ulist">
<ul>
<li>
<p>update <em>only</em> using <code>EclipseExternalLibraryWorkspace#updateState()</code></p>
</li>
<li>
<p>always mind that the diff of current state and cached state is a necessary information for cleaning dependencies of removed npms</p>
<div class="ulist">
<ul>
<li>
<p>see <code>EclipseExternalIndexSynchronizer#synchronizeNpms()</code> for implementation</p>
</li>
</ul>
</div>
</li>
<li>
<p>updating also happens when external root locations change (see ExternalIndexUpdater)</p>
</li>
</ul>
</div>
</li>
<li>
<p>Caching of user workspace projects (implemented in MuliCleartriggerCache)</p>
<div class="ulist">
<ul>
<li>
<p>caches only some project information and should be refactored along with Core, Model and EclipseBasedN4JSWorkspace</p>
</li>
</ul>
</div>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_dependency-injection"><a class="anchor" href="#_dependency-injection"></a><a class="link" href="#_dependency-injection">4.5. Dependency Injection</a></h3>
<div class="paragraph">
<p>There are some things to keep in mind when using dependency injection in the context of Xtext. This is a longer topic and it is discussed in the appendix
<a href="#sec:XtextInjection">Xtext Injection</a>.</p>
</div>
</div>
<div class="sect2">
<h3 id="_miscellaneous"><a class="anchor" href="#_miscellaneous"></a><a class="link" href="#_miscellaneous">4.6. Miscellaneous</a></h3>
<div class="ulist">
<ul>
<li>
<p>Resource load states: when an N4JS/N4JSD file is loaded, a certain sequence of processing is triggered (parsing,
linking, validation, etc.) and thus an <code>N4JSResource</code> transitions through a sequence of "load states". For details,
see <a href="#sec:N4JS_Resource_Load_States">N4JS Resource Load States</a>.</p>
</li>
</ul>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_parser"><a class="anchor" href="#_parser"></a><a class="link" href="#_parser">5. Parser</a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>Some of the concepts described here were presented at
<a href="https://www.youtube.com/watch?v=Xm-7aE1UMGY&amp;feature=youtu.be">EclipseCon 2013</a> and
<a href="https://vimeo.com/channels/xtextcon/98446435">XtextCon 2014</a>. Note that the material presented at the linked videos may be outdated.</p>
</div>
<div class="sect2">
<h3 id="sec:Parser_Overview"><a class="anchor" href="#sec:Parser_Overview"></a><a class="link" href="#sec:Parser_Overview">5.1. Overview</a></h3>
<div class="paragraph">
<p>The parser is created from an Xtext grammar. Actually, there are several grammars used as shown in <a href="#fig:cd_grammars">Figure CD Grammars</a>. These grammars and the parsers generated from them are described more closely in the following sections.</p>
</div>
<div id="fig:cd_grammars" class="imageblock center">
<div class="content">
<img src="chapters/05_parser/images/cd_grammars.svg" alt="cd grammars">
</div>
<div class="title">Figure 2. N4 Grammars</div>
</div>
</div>
<div class="sect2">
<h3 id="sec:N4JS_Parser"><a class="anchor" href="#sec:N4JS_Parser"></a><a class="link" href="#sec:N4JS_Parser">5.2. N4JS Parser</a></h3>
<div class="paragraph">
<p>One of the most tricky parts of JavaScript is the parsing because there is a conceptual mismatch between the <a href="#AC">ANTLR</a> runtime and the specified grammar. Another challenge is the disambiguation of regular expressions and binary operations. Both features require significant customizing of the generated parser (see figure below).</p>
</div>
<div id="fig:cd_ASIParser" class="imageblock center">
<div class="content">
<img src="chapters/05_parser/images/cd_ASIParser.svg" alt="cd ASIParser">
</div>
<div class="title">Figure 3. Overview custom parser implementation (runtime only)</div>
</div>
</div>
<div class="sect2 language-bash">
<h3 id="sec:Parser_Generation_Post_Processing"><a class="anchor" href="#sec:Parser_Generation_Post_Processing"></a><a class="link" href="#sec:Parser_Generation_Post_Processing">5.3. Parser Generation Post-Processing</a></h3>
<div class="paragraph">
<p>The ANTLR grammar that is generated by Xtext is post-processed to inject custom code into the grammar file before it is passed to the ANTLR tool. This is required in particular due to <a href="#AC">ASI</a> (Automated Semicolon Insertion), but for some other reasons as well.</p>
</div>
<div class="paragraph">
<p>Actually, there are several injections:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Due to Xtext restrictions, the generated ANTLR grammar file (<strong>*.g</strong>) is modified. This means that some some additional actions are added and some rules are rewritten.</p>
</li>
<li>
<p>Due to ANTLR restrictions, the generated ANTLR Java parser (<strong>*.java</strong>) os modified. This means that some generated rules are slightly modified to match certain requirements.</p>
</li>
<li>
<p>Due to Java restrictions, the generated Java parser needs to be preprocessed in order to reduce the size of certain methods since they must not exceed 64k characters. This is implemented by means of an MWE fragment, activated after the other post processing steps are done.</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>The first two steps are handled by <code>AntlrGeneratorWithCustomKeywordLogic</code>, which is configured with additional helpers in <code>GenerateN4JS.mwe2</code>. shows the customized classes which modify the code generation. These classes are all part of the <code>releng.utils</code> bundle.</p>
</div>
<div class="imageblock center">
<div class="content">
<img src="chapters/05_parser/images/cd_parsergeneration.svg" alt="cd parsergeneration">
</div>
<div class="title">Figure 4. Class Diagram Parser Generation</div>
</div>
<div class="sect3">
<h4 id="sec:Automatic_Semicolon_Insertion"><a class="anchor" href="#sec:Automatic_Semicolon_Insertion"></a><a class="link" href="#sec:Automatic_Semicolon_Insertion">5.3.1. Automatic Semicolon Insertion</a></h4>
<div class="paragraph">
<p>The EcmaScript specification mandates that valid implementations automatically insert a semicolon as a statement delimiter if it is missing and the input file would become invalid due to the missing semicolon. This is known as <a href="#AC">ASI</a>. It implies that not only valid implementations have to perform this, but a valid parser has to mimic this behavior in order to parse executable code. The <a href="#AC">ASI</a> is implemented by two different means.</p>
</div>
<div class="paragraph">
<p>The parser’s error recovery strategy is customized so it attempts to insert a semicolon if it was expected. Both strategies have to work hand in hand in order to consume all sorts of legal JavaScript code.</p>
</div>
<div class="sect4">
<h5 id="sec:Injected_code_in_the_Antlr_grammar_file"><a class="anchor" href="#sec:Injected_code_in_the_Antlr_grammar_file"></a><a class="link" href="#sec:Injected_code_in_the_Antlr_grammar_file">5.3.1.1. Injected code in the Antlr grammar file</a></h5>
<div class="paragraph">
<p>Under certain circumstances, the parser has to actively promote a token to become a semicolon even though it may be a syntactically a closing brace or line break. This has to happen before that token is consumed thus the rules for return statements, continue statements and break statements are enhanced to actively promote these tokens to semicolons.</p>
</div>
<div class="paragraph">
<p>The same rule is applied to promote line breaks between an expression and a possible postfix operator <code>++</code> or <code></code>. At this location the line break is always treated as a semicolon even though the operator may be validly consumed and produce a postfix expression.</p>
</div>
<div class="paragraph">
<p>In both cases, the method <code>promoteEOL()</code> is used to move a token that may serve as an automatically injected semicolon from the so called hidden token channel to the semantic channel. The hidden tokens are usually not handled by the parser explicitly thus they are semantically invisible (therefore the term hidden token). Nevertheless, they can be put on the semantic channel explicitly to make them recognizable. That’s implemented in the EOL promotion. The offending tokens include the hidden line terminators and multi-line comments that include line breaks. Furthermore, closing braces (right curly brackets) are included in the set of offending tokens as well as explicit semicolons.</p>
</div>
</div>
<div class="sect4">
<h5 id="sec:Customized_error_recovery"><a class="anchor" href="#sec:Customized_error_recovery"></a><a class="link" href="#sec:Customized_error_recovery">5.3.1.2. Customized error recovery</a></h5>
<div class="paragraph">
<p>Since the EOL promotion does not work well with Antlr prediction mode, another customization complements that feature. As soon as an invalid token sequence is attempted to be parsed and missing semicolon would make that sequence valid, an offending token is sought and moved to the semantic channel. This is implemented in the custom recovery strategy.</p>
</div>
</div>
</div>
<div class="sect3">
<h4 id="sec:_No_line_terminator_allowed_here__handling"><a class="anchor" href="#sec:_No_line_terminator_allowed_here__handling"></a><a class="link" href="#sec:_No_line_terminator_allowed_here__handling">5.3.2. Async and <code>No line terminator allowed here</code> Handling</a></h4>
<div class="paragraph">
<p>There is no way of directly defining <code>No line terminator allowed here</code>. This is required not only for <a href="#AC">ASI</a>, but also for <code>async</code>. This requires not only a special rule (using some rules from <a href="#sec:Automatic_Semicolon_Insertion">ASI</a>), but also a special error recovery since the token ’async’ may be rejected (by the manually enriched rule) which is of course unexpected behavior from the generated source code.</p>
</div>
</div>
<div class="sect3">
<h4 id="sec:Regular_Expression"><a class="anchor" href="#sec:Regular_Expression"></a><a class="link" href="#sec:Regular_Expression">5.3.3. Regular Expression</a></h4>
<div class="paragraph">
<p>The ANTLR parsing process can basically be divided into three steps. First of all, the file contents has to be read from disk. This includes the proper encoding of bytes to characters. The second step is the lexing or tokenizing of the character stream. A token is a basically a typed region in the stream, that is a triplet of token-id, offset and length. The last step is the parsing of these tokens. The result is a semantic model that is associated with a node tree. All necessary information to validate the model can be deduced from these two interlinked representations.</p>
</div>
<div class="imageblock center">
<div class="content">
<img src="chapters/05_parser/images/ad_parsing_simplified.svg" alt="ad parsing simplified">
</div>
<div class="title">Figure 5. Simplified visualization of the parsing</div>
</div>
<div class="paragraph">
<p>Since the default semantics and control flow of Antlr generated parsers do not really fit the requirements of a fully working JavaScript parser, some customizations are necessary. <strong>Regular expression literals in JavaScript cannot be syntactically disambiguated from div operations without contextual information.</strong> Nevertheless, the spec clearly describes, where a regular expression may appear and where it is prohibited. Unfortunately, it is not possible to implement these rules in the lexer alone, since it does not have enough contextual information. Therefore, the parser has been enhanced to establish a communication channel with the lexer. It announces when it expects a regular expression rather than a binary operation.</p>
</div>
<div class="paragraph">
<p>This required a reworking of the Antlr internals. Instead of a completely pre-populated <code>TokenStream</code>, the parser works on a lazy implementation that only reads as many characters as possible without a disambiguation between regular expression literals and divide operators.</p>
</div>
<div class="paragraph">
<p>Only after the parser has read this buffered tokens and potentially announced that it expects a regular expression, another batch of characters is processed by the lexer until the next ambiguous situation occurs. This is fundamentally different from the default behavior of Antlr.</p>
</div>
<div class="imageblock center">
<div class="content">
<img src="chapters/05_parser/images/sd_parsing_sequence.svg" alt="sd parsing sequence">
</div>
<div class="title">Figure 6. Abstract control and object flow during parsing</div>
</div>
<div class="paragraph">
<p>shows the involved classes which allow for this lexer-parser communication.</p>
</div>
<div class="imageblock center">
<div class="content">
<img src="chapters/05_parser/images/cd_parserlexercommunication.svg" alt="cd parserlexercommunication">
</div>
<div class="title">Figure 7. Class Diagram Parser-Lexer Communication</div>
</div>
</div>
<div class="sect3">
<h4 id="sec:Unicode"><a class="anchor" href="#sec:Unicode"></a><a class="link" href="#sec:Unicode">5.3.4. Unicode</a></h4>
<div class="paragraph">
<p>Unicode support in JavaScript includes the possibility to use unicode escape sequences in identifiers, string literals and regular expression literals. Another issue in this field is the specification of valid identifiers in JavaScript. They are described by means of unicode character classes. These have to be enumerated in the terminal rules in order to fully accept or reject valid or invalid JS identifiers.</p>
</div>
<div class="paragraph">
<p>For that purpose, a small code generator is used to define the terminal fragments for certain unicode categories. The <code>UnicodeGrammarGenerator</code> basically iterates all characters from <code>Character.MIN_VALUE</code> to <code>Character.MAX_VALUE</code> and adds them as alternatives to the respective terminal fragments, e.g. <code>UNICODE_DIGIT_FRAGMENT</code>.</p>
</div>
<div class="paragraph">
<p>The real terminal rules are defined as a composition of these generated fragments. Besides that, each character in an identifier, in a string literal or in a regular expression literal may be represented by its unicode escape value, e.g. ` u0060`. These escape sequences are handled and validated by the <code>IValueConverter</code> for the corresponding terminal rules.</p>
</div>
<div class="paragraph">
<p>The second piece of the puzzle are the unicode escaped sequences that may be used in keywords. This issue is covered by the <code>UnicodeKeywordHelper</code> which replaces the default terminal representation in the generated Antlr grammar by more elaborated alternatives. The keyword <code>if</code> is not only lexed as <code>’if’</code> but as seen in snippet
<a href="#lst:terminal_if">Terminal if listing</a>.</p>
</div>
<div id="lst:terminal_if" class="listingblock">
<div class="title">Terminal if</div>
<div class="content">
<pre class="highlight"><code>If :
( 'i' | '\\' 'u' '0`` 0`` 6`` 9' )
( 'f' | '\\' 'u' '0`` 0`` 6`` 6' );</code></pre>
</div>
</div>
</div>
<div class="sect3">
<h4 id="sec:Literals"><a class="anchor" href="#sec:Literals"></a><a class="link" href="#sec:Literals">5.3.5. Literals</a></h4>
<div class="paragraph">
<p>Template literals are also to be handled specially, see <code>TemplateLiteralDisambiguationInjector</code> for details.</p>
</div>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Modifiers"><a class="anchor" href="#sec:Modifiers"></a><a class="link" href="#sec:Modifiers">5.4. Modifiers</a></h3>
<div class="paragraph">
<p>On the AST side, all modifiers are included in a single enumeration <code>N4Modifier</code>. In the types model however, the individual modifiers are mapped to two different enumerations of <em>access</em> modifiers (namely <code>TypeAccessModifier</code> and <code>MemberAccessModifier</code>) and a number of boolean properties (in case of non-access modifiers such as <code>abstract</code> or <code>static</code>). This mapping is done by the types builder, mostly by calling methods in class <code>ModifierUtils</code>.</p>
</div>
<div class="paragraph">
<p>The grammar allows the use of certain modifiers in many places that are actually invalid. Rules where a certain modifier may appear in the AST are implemented in method isValid(EClass,N4Modifier) in class <code>ModifierUtils</code> and checked via several validations in <code>N4JSSyntaxValidator</code>. Those validations also check for a particular order of modifiers that is not enforced by the grammar.</p>
</div>
<div class="paragraph">
<p>See API documentation of enumeration <code>N4Modifier</code> in file <code>N4JS.xcore</code> and the utility class <code>ModifierUtils</code> for more details.</p>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Conflict_Resolutions"><a class="anchor" href="#sec:Conflict_Resolutions"></a><a class="link" href="#sec:Conflict_Resolutions">5.5. Conflict Resolutions</a></h3>
<div class="sect3">
<h4 id="sec:Reserved_Keywords_vs__Identifier_Names"><a class="anchor" href="#sec:Reserved_Keywords_vs__Identifier_Names"></a><a class="link" href="#sec:Reserved_Keywords_vs__Identifier_Names">5.5.1. Reserved Keywords vs. Identifier Names</a></h4>
<div class="paragraph">
<p>Keywords and identifiers have to be distinguished by the lexer. Therefore, there is no means to decide upfront whether a certain keyword is actually used as a keyword or whether it is used as an identifier in a given context. This limitation is idiomatically overcome by a data type rule for valid identifiers. This data type rule enumerates all keywords which may be used as identifiers and the pure IDENTIFIER terminal rule as seen in <a href="#lst:keywords_as_identifier">Keywords as Identifier listing</a>.</p>
</div>
<div id="lst:keywords_as_identifier" class="listingblock">
<div class="title">Keywords as Identifier</div>
<div class="content">
<pre class="highlight"><code class="language-ebnf" data-lang="ebnf">N4JSIdentifier: IDENTIFIER
| 'get'
| 'set'
...
;</code></pre>
</div>
</div>
</div>
<div class="sect3">
<h4 id="sec:Operators_and_Generics"><a class="anchor" href="#sec:Operators_and_Generics"></a><a class="link" href="#sec:Operators_and_Generics">5.5.2. Operators and Generics</a></h4>
<div class="paragraph">
<p>The ambiguity between shift operators and nested generics arises also from the fact, that Antlr lexer upfront without any contextual information. When implemented naively, the grammar will be broken, since a token sequence <code>a&gt;&gt;b</code> can either be part of <code>List&lt;List&lt;a&gt;&gt; b</code> or it can be part of a binary operation <code>int c = a &gt;&gt; b</code>. Therefore the shift operator may not be defined with a single token but has to be composed from individual characters (see <a href="#lst:shift_operator">Shift Operator listing</a>).</p>
</div>
<div id="lst:shift_operator" class="listingblock">
<div class="title">Shift Operator listing</div>
<div class="content">
<pre class="highlight"><code class="language-ebnf" data-lang="ebnf">ShiftOperator:
'&gt;' '&gt;' '&gt;'?
| '&lt;' '&lt;'
;</code></pre>
</div>
</div>
</div>
</div>
<div class="sect2 language-n4js">
<h3 id="sec:Content_Assist_Parser"><a class="anchor" href="#sec:Content_Assist_Parser"></a><a class="link" href="#sec:Content_Assist_Parser">5.6. Content-Assist Parser</a></h3>
<div class="admonitionblock warning">
<table>
<tr>
<td class="icon">
<i class="fa icon-warning" title=""></i>
</td>
<td class="content">
This section may be outdated!
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>The <a href="#AC">CA</a> parser also needs adjustments for supporting automatic semicolon insertion and regular expressions. Instead of modifying the <a href="#AC">CA</a> parser generator similar to the normal parser, the former reuses parts of the latter as far as possible. That is, the token sequence that is produced during production parsing is used as is for the content assist parser. Semicolons have already been inserted where appropriate and regular expression are successfully distinguished from divide operators.</p>
</div>
<div class="paragraph">
<p>Since the n4js grammar uses syntactic predicates, the content assist parser is compiled with backtracking enabled. This is always the case for Xtext’s CA parsers that rely on backtracking or predicates (local backtracking) in the production parser. This approach is both good (CA works in general) and bad (unpredictable decisions in case of error at locations prior to the cursor). Since parsing with backtracking enabled makes for a fundamental difference in how the prediction and parsing works and how the parser decides which decision paths to take, the customization patterns from the production parser are not applied 1:1 to the CA parser, but adapted instead. The content assist parser doesn’t use a freshly lexed token stream with unicode support, ASI or regular expression literals, but instead uses a synthesized token sequence which is rebuilt from the existing node model.</p>
</div>
<div class="paragraph">
<p>The token stream that is consumed by the content assist parser is therefore not created by a lexer but by the <code>org.eclipse.n4js.ui.contentassist.NodeModelTokenSource</code>.
It traverses the existing node model that is contained in the resource and was produced by the production parser. This approach has the significant advantage that any decision that was made by that parser is also immediately applicable to the content assist infrastructure. For that purpose, the leaf nodes of the node model are mapped to ANTLR token types.
This is achieved by the <code>org.eclipse.n4js.ui.contentassist.ContentAssistTokenTypeMapper</code> which is capable to provide the untyped ANTLR token type (primitive int) for a given grammar element.</p>
</div>
<div class="paragraph">
<p>Special considerations have been made for the last token in the produced source. If it overlaps with an existing leaf node but does not fully cover it, the plain Antlr lexer is used to consume the prefix that is overlapping. Since the terminals will never overlap with each other the longest match always wins without backtracking in the lexer, it is save to assume that only one token is produced from the prefix. The very last token in the <code>org.eclipse.n4js.ui.contentassist.NodeModelTokenSource</code> is always the EOF token (<code>org.antlr.runtime.Token.EOF_TOKEN</code>).</p>
</div>
<div class="paragraph">
<p>Given that the token source is equal to the prefix in the production token source, some more thought has to be put into the synthesized end of file. The production parser used the complete file to decide where to automatically insert a semicolon and where not to. This would potentially change if there was another token next to the artificial EOF. Therefore, two cases have to considered. The first one describes CA request next to an automatically inserted semicolon and the second one describes CA requests at a position where a semicolon could have been inserted if the token to the right was another one. The <code>org.eclipse.n4js.ui.contentassist.CustomN4JSParser</code> reflects these cases. Heuristics are applied to the end of the token sequence to decide whether a second pass has to be performed to collect yet more following elements. Based on the concrete sequence, the last automatically inserted semicolon is removed from the sequence prior to the second pass or such is a token is explicitly synthesized and appended. Besides the second pass, another special treatment is made for postfix expressions. Those may not be interrupted by a hidden semicolon so those are filtered from the resulting follow set if appropriate.</p>
</div>
<div class="paragraph">
<p>The parser is used by the <code>org.eclipse.n4js.ui.contentassist.ContentAssistContextFactory</code> where all relevant entry points from the super class are specialized to pass the node model in the the parser facade (<code>org.eclipse.n4js.ui.contentassist.CustomN4JSParser</code>). In that sense, the ContentAssistContextFactory serves as a drop-in replacement binding the default <code>ParserBasedContentAssistContextFactory.StatefulFactory</code>.</p>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_type-system"><a class="anchor" href="#_type-system"></a><a class="link" href="#_type-system">6. Type System</a></h2>
<div class="sectionbody">
<div class="sect2 language-n4js">
<h3 id="sec:Type_Model_and_Grammar"><a class="anchor" href="#sec:Type_Model_and_Grammar"></a><a class="link" href="#sec:Type_Model_and_Grammar">6.1. Type Model and Grammar</a></h3>
<div class="paragraph">
<p>The type model is used to define actual types and their relations (meta-model is defined by means of Xcore in file <code>Types.xcore</code>)
and also references to types (meta-model in <code>TypeRefs.xcore</code>). The type model is built via the <code>N4JSTypesBuilder</code> when a resource
is loaded and processed, and most type related tasks work only on the type model. Some types that are (internally) available
in N4JS are not defined in N4JS, but instead in a special, internal type language not available to N4JS developers, called N4TS
and defined in file <code>Types.xtext</code>.</p>
</div>
<div class="paragraph">
<p>The types are referenced by AST elements; vice versa the AST elements can be referenced from the types (see <code>SyntaxRelatedTElement</code>).
This backward reference is a simple reference to an EObject.</p>
</div>
<div class="sect3">
<h4 id="sec:Type_Model_Overview"><a class="anchor" href="#sec:Type_Model_Overview"></a><a class="link" href="#sec:Type_Model_Overview">6.1.1. Type Model Overview</a></h4>
<div class="paragraph">
<p>The following figure, <a href="#fig:cd_typeAndTypeRefHierarchy">Types and Type References</a>, shows the classes of the type model and their inheritance relations, both the actual type definitions as defined in <code>Types.xcore</code> and the type references defined in <code>TypeRefs.xcore</code>. The most important type reference is the <code>ParameterizedTypeRef</code>; it is used for most user-defined references, for both parameterized and non-parameterized references. In the latter case, the list of type arguments is empty.</p>
</div>
<div id="fig:cd_typeAndTypeRefHierarchy" class="imageblock">
<div class="content">
<img src="chapters/06_typesystem/images/cd_typeModelHierarchy_allInOne.png" alt="cd typeModelHierarchy allInOne">
</div>
<div class="title">Figure 8. Type Model Overview: Types in the upper half and Type References in the lower half.</div>
</div>
<div class="paragraph">
<p>Most types are self-explanatory. <code>TypeDefs</code> is the container element used in N4TS. Note that not all types and properties of types are available in N4JS – some can only be used in the N4TS language or be inferred by the type system for internal purposes. Some types need some explanation:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>TObjectPrototype</code>: Metatype for defining built-in object types such as <code>Object</code> or <code>Date</code>, only available in N4TS.</p>
</li>
<li>
<p><code>VirtualBaseType</code>: This type is not available in N4JS. It is used to define common properties provided by all types of a certain metatype. E.g., it is used for defining some properties shared by all enumerations (this was the reason for introducing this type).</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>We distinguish four kinds of types as summarized in <a href="#tab:KindOfTypes">Kind Of Types</a>. Role is an internal construct for different kind of users who can define the special kind of type. The language column refers to the language used to specify the type; which is either N4JS or N4TS.</p>
</div>
<table id="tab:KindOfTypes" class="tableblock frame-all grid-all spread">
<caption class="title">Table 1. Kind of Types</caption>
<colgroup>
<col style="width: 10%;">
<col style="width: 10%;">
<col style="width: 10%;">
<col style="width: 70%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Kind</th>
<th class="tableblock halign-left valign-top">Language</th>
<th class="tableblock halign-left valign-top">Role</th>
<th class="tableblock halign-left valign-top">Remark</th>
</tr>
</thead>
<tbody>
<tr>
<th class="tableblock halign-left valign-top"><p class="tableblock">user</p></th>
<th class="tableblock halign-left valign-top"><p class="tableblock">N4JS</p></th>
<th class="tableblock halign-left valign-top"><p class="tableblock">developer</p></th>
<td class="tableblock halign-left valign-top"><p class="tableblock">User defined types, such as declared classes or functions. These types are to be explicitly defined or imported in the code.</p></td>
</tr>
<tr>
<th class="tableblock halign-left valign-top"><p class="tableblock">library</p></th>
<th class="tableblock halign-left valign-top"><p class="tableblock">N4JSD</p></th>
<th class="tableblock halign-left valign-top"><p class="tableblock">developer</p></th>