| EASE AutoCompletion Readme |
| ========================== |
| |
| Table of contents: |
| ------------------ |
| 1. Introduction |
| 2. Overview |
| 3. Code Explanation |
| 3.1. Extension Points |
| 3.1.1. completionProcessor |
| 3.1.2. language#completionAnalyzer |
| 3.2. Interfaces and Classes |
| 3.2.1. CompletionProviderDispatcher |
| 3.2.2. ICompletionProvider |
| 3.2.3. AbstractCompletionProvider |
| 3.2.4. ICompletionContext |
| 3.2.5. CompletionContext |
| 3.2.6. ICompletionSource |
| 3.2.7. ICompletionAnalyzer |
| 3.2.8. AbstractCompletionAnalyzer |
| 3.2.9. JavaScriptEditorCompletionComputer |
| 4. Completion Flow |
| 5. List of Completion Providers |
| 5.1. LoadedModuleCompletionProvider |
| 5.2. ScriptShellCompletionProvider |
| 5.3. SourceStackCompletionProvider |
| 5.4. KeywordCompletionProvider |
| 5.5. CustomAnnotationCompletionProvider |
| 5.6. JavaLookupCompletionProvider |
| 6. Open Issues |
| 6.1. All code is single-threaded |
| 6.2. No caching mechanism |
| 6.3. No mechanism to detect duplicates |
| 6.4. No scalable recursion mechanism for e.g. included files |
| |
| |
| 1. Introduction |
| --------------- |
| The purpose of this document is to give an overview on the current auto-completion features of EASE. |
| It gives an overview of the involved classes and extension points as well as the flow of auto-completion. |
| |
| |
| 2. Overview |
| ----------- |
| Eclipse currently does not support a single interface for code completion. |
| Different text editors and the ScriptShell all have different extension points. |
| Therefore new extension points and interfaces have been defined to allow central calculation of proposals for EASE components. |
| These proposals are then wrapped to match the different interfaces. |
| |
| Currently code completion is available for the ScriptShell and the JavaScript editor. |
| As this document will explain adding new completion providers for other editors or scripting languages should be easy. |
| |
| The main class involved is the 'CompletionProviderDispatcher' (see 3.2.1.). |
| It dispatches calculation of an ICompletionContext(3.2.4.) with relevant information based on the input. |
| It then dispatches this context to calculate the actual completion proposals. |
| Developers that are interested in the code should use this class as their entry-point. |
| |
| The benefits of the current implementation are: |
| - Single generic interface for completion providers |
| + Reusable for both the ScriptShell and editors. |
| + (Possibly) Reusable for different scripting languages. |
| - Single entry point 'CompletionProviderDispatcher' to calculate proposals |
| - New completion providers can be implemented very fast |
| - Completion support for new scripting languages can be added fast |
| - Used for type detection in code |
| + Necessary for completion calculation |
| + Helpful for utility functionality like hover tool-tips |
| |
| |
| 3. Code Explanation |
| ------------------- |
| The following sections will be excerpts from the code documentation or short descriptions of code. |
| They should help explain what the involved parties are, what they are intended to do and most of all they should help understand the other parts of this document. |
| Note that they are meant as short introductions and that further documentation is available via the JavaDocs. |
| |
| 3.1. Extension Points |
| ````````````````````` |
| This section will give an overview on the involved extension points. |
| |
| 3.1.1. completionProcessor |
| '''''''''''''''''''''''''' |
| Plugin: org.eclipse.ease.ui |
| Schema: org.eclipse.ease.ui/schema/completionProcessor.exsd |
| Description: This extension point is used to register new completion providers. |
| These completion providers are used to actually calculate completion proposals. |
| completionProcessors must have an attribute 'class' that maps to the corresponding 'ICompletionProvider' (see 3.2.2.). |
| An optional engine ID can be given to restrict where this provider is applied. If empty the completionProcessor will be used for all languages. |
| |
| 3.1.2. language#completionAnalyzer |
| '''''''''''''''''''''''''''''''' |
| Plugin: org.eclipse.ease |
| Schema: org.eclipse.ease/schema/language.exsd |
| Description: For code-completion only the 'completionAnalyzer' element is used. |
| The attribute 'class' maps to the corresponding class of 'ICompletionAnalyzer' type (see 3.2.7.). |
| This class is used to statically analyze a given line of code. |
| This includes: |
| - Extracting the relevant portion of the code |
| - Parsing the code to a more suitable 'ICompletionContext' (see 3.1.3.) |
| The element also includes the attribute 'engineID' to decide for which engine to use this analyzer. |
| |
| |
| 3.2. Interfaces and Classes |
| ``````````````````````````` |
| |
| The following list will give short introductions to the involved interfaces classes. |
| It will quickly describe the most relevant methods that will hopefully help clarify the completion flow later. |
| Again, for further information please refer to the JavaDocs. |
| |
| 3.2.1. CompletionProviderDispatcher |
| ''''''''''''''''''''''''''''''''''' |
| Package: |
| org.eclipse.ease.ui.completion |
| Description: |
| This is the main entry point for completion calculation. |
| It handles the creation of an ICompletionContext for given input and the calculation of actual proposals. |
| It implements the IContentProposalProvider interface to be directly usable with ScriptShell. |
| Methods of interest: |
| - void setAnalyzer(ICompletionAnalyzer) |
| Sets the 'ICompletionAnalyzer' (see 3.2.3) used to calculate static completion context for given input. |
| - void (un)registerCompletionProvider(ICompletionProvider) |
| Adds or removes a new ICompletionProvider to/from the internal list. |
| - void setScriptEngine(IScriptEngine) |
| Sets the script engine in use (also used to indicate ScriptShell use) |
| - void addCode(String code): |
| Dispatches code to addCode(String) for all registered ICompletionProviders . |
| - ICompletionContext getContext(String) |
| Calculates ICompletionContext for the given input. |
| First uses ICompletionAnalyzer to get initial static context. |
| Then uses ICompletionProvider#refineContext(ICompletionContext) to get a refined context. |
| - Collection<ICompletionSource> calculateProposals(ICompletionContext) |
| Calculates all proposals for the given ICompletionContext using ICompletionProvider#getProposals(ICompletionContext). |
| - Collection<ICompletionProvider> getProviders(String) |
| Gets all completion providers registered via extension point that match given engine description. |
| See: 3.1.1. |
| |
| 3.2.2. ICompletionProvider |
| '''''''''''''''''''''''''' |
| Package: |
| org.eclipse.ease.ui.completion |
| Description: |
| Interface used to refine ICompletionContext and create completion proposals. |
| Each ICompletionProvider should focus on a single completion suggestion type. |
| Examples: KeywordCompletionProvder, CustomAnnotationCompletionProvider |
| Methods of interest: |
| - void addCode(String) |
| Adds a given piece of code. Providers can try to extract relevant information. |
| - ICompletionContext refineContext(ICompletionContext) |
| Tries to refine given context. |
| This refinement is focused on updating the sourceStack and get matching ICompletionSources for each object in chain. |
| Most of the times it is only necessary to correctly identify first item in chain. |
| - Collection<ICompletionSource> getProposals(ICompletionContext) |
| Calculates proposals for given context. |
| Checks sourceStack and filter to only get real matches. |
| |
| 3.2.3. AbstractCompletionProvider |
| ''''''''''''''''''''''''''''''''' |
| Package: |
| org.eclipse.ease.ui.completion |
| Description: |
| Base class for all completion providers. |
| It is highly recommended that you subclass this for each new ICompletionProvider. |
| |
| 3.2.4. ICompletionContext |
| ''''''''''''''''''''''''' |
| Package: |
| org.eclipse.ease.completion |
| Description: |
| Interface to store information about completion context. |
| Most of the time the simple data-storage implementation CompletionContext (3.2.5.) should suffice. |
| Methods of interest: |
| - String getInput() |
| Returns the initial input the context is based on. |
| - String getFilter() |
| Returns the filter to be applied to all proposals. |
| Examples: foo -> foo |
| foo.bar.baz -> baz |
| - List<ICompletionSource> getSourceStack() |
| Returns the source stack of the context. |
| The source stack is a list of all items in a call chain. |
| This source stack can be used to get to the class of interest for completion calculation. |
| Examples: foo -> [] |
| foo.bar.baz -> [foo, bar] |
| |
| 3.2.5. CompletionContext |
| '''''''''''''''''''''''' |
| Package: |
| org.eclipse.ease.completion |
| Description: |
| Simple data-storage implementation of ICompletionContext. |
| Offers static method to "follow" source stack and refine it. |
| Methods of interest: |
| - static List<ICompletionSource> refineSourceStack(List<ICompletionSource) |
| Should be called when first item in source stack has been refined / identified. |
| Iterates over all following items and gets information based on parent class. |
| |
| 3.2.6. ICompletionSource |
| '''''''''''''''''''''''' |
| Package: |
| org.eclipse.ease.completion |
| Description: |
| Interface to get information about single item in call chain. |
| Also used to store information about completion proposal. |
| Stores all relevant information to sufficiently describe a single item in the chain. |
| Most of the time the simple data-storage implementation org.eclipse.ease.completion.CompletionSource should suffice. |
| Methods of interest: |
| - SourceType getSourceType() |
| Returns the source type of the item. |
| - String getName() |
| Returns the name of the item. |
| - Class<?> getClazz() |
| Returns the class the item belongs to. |
| - Object getObject() |
| Returns the actual object (might be null if other information is enough) |
| - String getDescription() |
| Gets a description of the item. |
| Can be used to display e.g. help tips. |
| Example: |
| java.lang.StringBuilder.toString() |
| Source type: JAVA_METHOD |
| Name: toString |
| Class: java.lang.StringBuilder |
| Object: Method toString |
| Description: javadoc of toString method |
| |
| |
| 3.2.7. ICompletionAnalyzer |
| '''''''''''''''''''''''''' |
| Package: |
| org.eclipse.ease.completion |
| Description: |
| Interface to get static information about given piece of code. |
| Helps parse the input to simplify context creation for ICompletionProviders. |
| Methods of interest: |
| - ICompletionContext getContext(String, int) |
| Returns static context information for given piece of code. |
| This includes removing everything that is not necessary for completion calculation. |
| Distinguishing between constructors, calls and fields for items in call chain. |
| Actually retrieving the matching call-chain. |
| The returned context is then provided to all completion providers for refinements. |
| - String getIncludedCode(String, Object) |
| Returns the given code with all included code added in the position it belongs. |
| Optional parent object given to have a root for relative imports |
| |
| 3.2.8. AbstractCompletionAnalyzer |
| ''''''''''''''''''''''''''''''''' |
| Package: |
| org.eclipse.ease.completion |
| Description: |
| Abstract base class for ICompletionAnalyzers. |
| All ICompletionAnalyzers should subclass this to simplify their work. |
| Actually implements all the interface methods but internally calls simpler abstract methods. |
| Methods of interest: |
| - abstract String removeUnnecessaryCode(String) |
| Abstract method to remove all unnecessary code from given input. |
| Since most of the actual parsing is generic only unnecessary code is language specific. |
| - List<String> splitChain(String) |
| Splits a given piece of code to call chain (of Strings). |
| Used because different scripting languages might have different object accessor. |
| - abstract ICompletionSource toCompletionSource(String) |
| Parses a given piece of code to ICompletionSource. |
| The given string will only contain information about single item in call chain. |
| Used because different scripting languages might have different syntax. |
| - abstract Patter getIncludePattern() |
| Returns regular expression to query for includes. |
| The pattern must have a single group with the string that needs to be included. |
| Used in getIncludedCode(String, Object) to have generic base for most languages. |
| |
| 3.2.9. JavaScriptEditorCompletionComputer |
| ''''''''''''''''''''''''''''''''''''''''' |
| Package: |
| org.eclipse.ease.lang.javascript.ui.completion |
| Description: |
| Completion computer for JavaScript editor. |
| Internally uses only CompletionProviderDispatcher to calculate the proposals. |
| Since (as mentioned) eclipse does not have a single interface for code completion wrapper classes need to be implemented for all different editors. |
| Methods of interest: |
| - List<?> computeCompletionProposals(ContentAssistInvocationContext, IProgressMonitor) |
| Sample implementation of how CompletionProviderDispatcher can be used to create proposals for different editors. |
| Implements IJavaCompletionProposalComputer interface to be usable with .js files. |
| |
| |
| 4. Completion Flow |
| ------------------ |
| This section is intended to explain what is actually happening when auto-completion is triggered. |
| The flow is explained in a generic way but differences between the ScriptShell and the JavaScript editor are still noted. |
| |
| First off the CompletionProviderDispatcher needs to be instantiated and set up. |
| For the ScriptShell this happens in the addAutoCompletion() method whereas for the JavaScript editor this happens in the constructor the first time auto-completion is triggered. |
| Based on the currently selected script type/engine the corresponding ICompletionProviders are retrieved via the getProviders(String) method. |
| These providers are then added and (if available) the ScriptEngine is set. |
| Further a suiting ICompletionAnalyzer needs to be set. This is either done via the setAnalyzer(ICompletionAnalyzer) method or implicitely via setScriptEngine(IScriptEngine). |
| With these steps the CompletionProviderDispatcher is ready to calculate proposals. |
| |
| During normal usage each line entered in the ScriptShell will be passed to all registered CompletionProviders. |
| With the JavaScript editor the code is only added when auto-completion is triggered. |
| Each completion provider needs to store the matching information for later use. |
| Completion providers may also choose to ignore all given code (e.g.: Keyword providers do not user input). |
| |
| Once the user presses the completion keybinding (default: CTRL+SPACE) the actual completion calculation is started. |
| |
| As a first step all necessary code must be gathered. |
| For the ScriptShell this is only the current input but for the JavaScript editor this also can also be code from included files. |
| Therefore the ICompletionAnalyzer#getIncludedCode(String) is used to retrieve all code including includes as a single String. |
| |
| The gathered code will then be used to calculate the corresponding completion context. |
| First a static completion context for the given language will be calculated using ICompletionAnalyzer#getContext(String, int). |
| This step consists of removing all unnecessary code, splitting the remaining code into a call chain and extracting the filter. |
| |
| The calculated static context will then be refined by all registered completion providers using their refineContext(ICompletionContext) method. |
| In almost all cases this method will only try to identify the first item in the call chain and refine the rest using CompletionContext#refineSourceStack(List<ICompletionSource). |
| If successful this context will contain all information about the items prior in the call-chain. |
| |
| This context is then used to calculate the actual proposals using the CompletionProviderDispatcher#calculateProposals(ICompletionContext) method. |
| Internally all registered completion providers get the context and add matching proposals based on the filter and the call-chain. |
| |
| Once the proposals are calculated they need to be parsed to the correct type. For the ScriptShell this would be an IContentProposal[] whereas for JavaScript it would be a List<?>. |
| |
| The "casted" proposals are then used by all different completion mechanisms to get include the desired input. |
| |
| |
| 5. List of Completion Providers |
| ------------------------------- |
| The following list will explain what completion providers are currently available and what they are used for. |
| |
| 5.1. LoadedModuleCompletionProvider |
| ``````````````````````````````````` |
| Interface: ScriptShell, Editors |
| Languages: All |
| Description: |
| Detects when a module has been loaded. |
| Offers information about methods and fields of said module. |
| |
| 5.2. ScriptShellCompletionProvider |
| `````````````````````````````````` |
| Interface: ScriptShell |
| Language: All |
| Description: |
| Simple completion provider to query given script engine if available. |
| Can be used for local variables, etc. |
| |
| 5.3. SourceStackCompletionProvider |
| `````````````````````````````````` |
| Interface: ScriptShell, Editors |
| Languages: All |
| Description: |
| Used for non-empty call chain. |
| Calculates all matches based on class of last element and filter. |
| Needs a refined context to work. |
| |
| 5.4. KeywordCompletionProvider |
| `````````````````````````````` |
| Interface: ScriptShell, Editors |
| Languages: All (needs different keywords for each langauge) |
| Description: |
| Simply provides a list of keywords for selected language. |
| |
| 5.5. CustomAnnotationCompletionProvider |
| ``````````````````````````````````````` |
| Interface: Editors |
| Languages: All |
| Description: |
| Parses code for EASE custom type annotation. |
| Annotation constists of identifier, variable name and variable type. |
| For this to work annotation must be put in commend (otherwise language would throw syntax error) |
| Syntax: @type varName varClass |
| Example: @type foo java.lang.String |
| |
| 5.6. JavaLookupCompletionProvider |
| ````````````````````````````````` |
| Interface: ScriptShell, Editors |
| Languages: All |
| Description: |
| Used to try to evaluate constructor based on java class name. |
| Based on this information call chain can be evaluated easily. |
| Relies on Class.forName(String) Java method to work. |
| |
| |
| 6. Open Issues |
| -------------- |
| The following list will give an overview what is currently not supported or might need refactoring. |
| |
| 6.1. All code is single-threaded |
| ```````````````````````````````` |
| Calculations might take too long and UI responsiveness could decrease. |
| All registered completion providers are currently called sequentially, would be faster if called in parallel. |
| At the moment not a problem but might be once more providers are registered. |
| |
| 6.2. No caching mechanism |
| ````````````````````````` |
| Code might be unchanged but will still be parsed. |
| This is especially bad for editors because lots of unnecessary code might be parsed for each suggestion. |
| |
| 6.3. No mechanism to detect duplicates |
| `````````````````````````````````````` |
| The same suggestion might be proposed several times (e.g. from loaded module and from available module) |
| |
| 6.4. No scalable recursion mechanism for e.g. included files |
| ```````````````````````````````````````````````````````````` |
| Each language can implement ICompletionAnalyzer#getIncludedCode(String) but it is currently not possible to dynamically add completion providers that also add code during proposal calculation. |