query/plugins/org.eclipse.acceleo.query.doc/input/index.textile

h2. Introduction

The Acceleo Query Language (AQL) is a language used to navigate and query an EMF model. In this document, you will find the description of the syntax, all the services and the standard library of AQL.
AQL as a query engine is small, simple, fast, extensible and it brings a richer validation than the MTL interpreter.

For those looking for a simple and fast interpreters for your EMF models, AQL can provide you with a lot of features, including:

* Support for static and dynamic Ecore models, no query compilation phase required.
* The least possible overhead at evaluation time. During this phase, the evaluation goes forward and will not even try to validate or compile your expressions. Errors are tracked and captured along the way.
* Strong validation: types are checked at validation time and the metamodels used are analyzed to do some basic type inference and avoid false positive errors.
* Union types: In some context, a variable in a given query can have N potential types with N being often greater than one. AQL can embrace this fact without having to fall back to EObject as soon as there is more than one type.
* A simple and straightforward implementation easily extensible with Java classes providing extension methods.
* A very narrow dependency surface: AQL uses the very central parts of EMF, Guava and Antlr so that we could easily deploy AQL outside of Eclipse in a server or a standalone scenario.

The AQL interpreter is used in Sirius with the prefix "aql:".

h2. Syntax

h3. Basics

The syntax is very similar to the OCL syntax. An expression always starts with a variable
@aVariable@

The variable named *self* represent the current object (think of it as the @this@ in Java).

Let's consider the following metamodel : 

!../assets/pics/family_ecore.png! 

From a variable one can access field or reference values using the @.@ separator.
With @self@ being an instance of *Person*, @self.name@ returns the value of the attribute *name* and @self.father@ return the father of the person.

If the attribute or the reference is multi-valued, then @self.parents@ will return a collection.

Calls can be chained, as such @self.parents.name@ will return a collection containing the names of the parents.

If one want to access the collection itself, then the separator @->@ must be used, as such @self.parents.name->size()@ will return the number of elements in the collection whereas @self.parents.name.size()@ will return a collection containing 
the sizes of each name.  

AQL can also call methods modeled as EOperations or defined through Java services. The syntax denoting such a call is @.@ for instance @self.someCall()@ will call the *someCall* method and return the result.

h3. Working with collections

Filtering a collection is generaly done using either @->filter(..)@ to keep elements of a given type or @->select(..)@ to keep elements which are validating a given condition.

With @self@ being an instance of *Family*, @self.members->filter(family::Man)@  will return all the members of the family which are mens and @self.members->select( p | p.name.startsWith('A'))@ will return all the members of the family which have a name starting by the letter 'A'.

To access an element at a particular index you can use the operation @->at(..)@; @self.members->at(1)@ will return the first person which is a member of the family (in that specific case it is probably better to use @self.members->first()@

AQL has two kinds of collections, a @Sequence@ which is a list, or an @OrderedSet@ which does not allow doubles. You can convert a @Sequence@ to an @OrderedSet@ by as such : @self.members->asSet()@

You can also define a collection by extension using the following syntax: 
* @OrderedSet{self}@ which returns a set containing the current EObject.
* @Sequence{self, self.eContainer()}@ returns a sequence containing the current EObject and its parent.


h3. Navigation axes

AQL provides operations out of the box to browse the model. Most notably :
* @self.eContainer()@ returns the parent of the current object if there is one.
* @self.eAllContents(some::Type)@ returns all direct and indirect children matching the given type.
* @self.eContents()@ return all the direct children.
* @self.eInverse('father')@ returns the cross reference of the reference named 'father'. In this case it will return all the persons which have the current object (self) as a father.

h3. Conditions

AQL provides an *If* but it has to be an expression and not a statement. As such one *has to define the else*. Here is the syntax

*if* @self.name.startsWith('a')@ *then* @self@ *else* @self.eContainer()@ *endif* 

h3. Walkthrough using an UML example

Let's move to a slightly more complex example to learn how to navigate through a model. We will work with a model instance from the following metamodel: a simplified view of the UML2 metamodel with *Package*, *UseCase*, *Model* or *Component* instances. 

!../assets/pics/simple-uml-ecore.jpg! 


The following illustration demonstrate the result of the just typing @self@ as a query. At the bottom every instance of the UML model is represented by a node with containment relationships from top to bottom and displaying the non-contained references in between those nodes through horizontal edges. The result of the query is highlighted in *blue*. 

!../assets/pics/self.jpg!  

The variable *self* here is the *Class* named "Invoice" in the model, as such the query @self@ hightlight this instance.

h4. eContainer()

!../assets/pics/self-econtainer.jpg!

When using the query @self.eContainer()@ the cursor move from the @self@ variable to its most direct parent, here the *Component* instance named "Accounting".

!../assets/pics/self-econtainer-econtainer.jpg!

Such calls can be chained and as such @self.eContainer().eContainer()@ return the *Component* parents: the *Package* named "Components".

!../assets/pics/self-econtainer-model.jpg!

The @eContainer()@ call can also be used with a type parameter, in that case it will be transitively executed up to the point where an instance of the given type is found.

In this case then @self.eContainer(uml::Model)@ goes up to the root of the graph. If no instance of the given type is found in the parents then the query returns an empty result.  

!../assets/pics/self-econtainer-class.jpg!

@eContainer()@ or any other service using types as parameters will match both the given types or its subtypes. The only exception to this rule is the @oclIsTypeOf(..)@ service which is intended to check only for the given type and not its subtypes.  

When using the query @self.eContainer(uml::Class)@ the result is an instance of *Component* as the *Component* type extends *Class* in the metamodel.

!../assets/pics/self-econtainerorself.jpg!

A variant of @eContainer(..)@ named @eContainerOrSelf(..)@ is provided it will first check the type of the current instance. As such the query @self.eContainerOrSelf(uml::Class)@ when *self* is the "Invoice" class returns this instance. 
  

h4. eContents()

!../assets/pics/self-econtents.jpg!

One use @eContainer()@ to go up in the parent. The @eContents()@ axes is its counterpart and returns the direct children of the element.

h4. select()

!../assets/pics/self-econtents-select-name-id.jpg!

The @select(...)@ service can be used to filter elements from a list by veryfing a predicate. In the query @self.eContents()->select(p | p.name = 'id')@ the query only returns the elements which have a name equal to *"id"*, in this case a single element.


!../assets/pics/self-econtents-select-name-notid.jpg!

Equality is checked with a single @=@, inequality is expressed with the operator @<>@.

!../assets/pics/self-econtents-select-visibility.jpg!

Comparing values with an enumeration is slightly different as the enumeration value should be explicitely qualified. In the @self.eContents()->select(p | p.visibility =  uml::VisibilityKind::private )@ query the expression @uml::VisibilityKind::private@ denotes the enumeration literal named *private* which is contained in the *VisibilityKind* enumeration of the *uml* metamodel.

h4. eAllContents()

!../assets/pics/self-econtainer-model-eallcontents.jpg!

The @eAllContents()@ axe is used to browse direct and indirect children. It goes from the starting point to the leafs of the model. Here the expression starts with @self.eContainer(uml::Model)@ which has we've seen before goes up until an instance of *Model* is found. From here @eAllContents@ is executed returning all the direct and indirect childrens of the "Travel Agency"  *model* instance.

!../assets/pics/econtainer-eallcontents-select-name-startswith.jpg!

Just like any other collection it can be filtered to retrieve, for instace, the elements whose name is starting by the letter "I".

!../assets/pics/self-econtainer-model-eallcontents-components.jpg!

A type parameter can be used to retrieve the direct or indirect children of a specific type: here *components*.

!../assets/pics/self-econtainer-model-eallcontents-usecases.jpg!

Or *use case* instances.

!../assets/pics/self.econtainer-model-eallcontents-multipletypes.jpg!

It is interesting to note that the parameter can also be a collection of types, enabling the retrieval of many elements through a single pass.  


h4. eInverse()

Queries using @eAllContents@ must be designed with care as they tend to lead to an intense browsing of the model. In many cases they can be replaced with @eInverse()@ calls to retrieve elements of interests.

!../assets/pics/self-econtainer-einverse.jpg!

@eInverse()@ returns any element which as a relationship with the current one. This relationship can be indifferently a containment one or not.

!../assets/pics/econtainer-einverse-usecase.jpg!

It is often of interest to restrict the type of elements we expect out of the @eInverse()@ call. With the query @self.eContainer().eInverse(uml::UseCase)@ only use cases instances will be returned, here the *UseCase* named "to Invoice" which refers to the "Accounting" *Component* through the reference named *subject*

!../assets/pics/self-econtainer-einverse-packagedelement.jpg!

One can also be even more explicit and query for a specific reference name, here @packagedElement@ : only the *Package* named "Components" refers to the "Accounting" *Component* through the reference named "packagedElement".


h2. Language Reference

###.  The actual reference documentation will be appended to 'These sections are listing all the services of the standard library of AQL' at build time. See DocumentationGenerator.java

These sections are listing all the services of the standard library of AQL.

h3. Syntax Reference

h4. References

|<. _variable_name_ |a reference to a variable| myVariable |
|<. _expression_ *.* _feature_name_ |implicit collect| eClass.name |
|<. _expression_ *.* _service_name_*(* ( _expression_ (*,* _expression_ ) * ) ? *)* |implicit collect| myVariable.toString() |
|<. _expression_ *->* _service_name_*(* ( _expression_ (*,* _expression_ ) * ) ? *)* |call on the collection itself if the expression is not a collection it will be wrapped into an ordered set| mySequence->sep(',') |

h4. Operators

|<. *not* _expression_ |call the not service| not eClass.interface |
|<. *-* _expression_ |call the unaryMin service| -3 |
|<. _expression_ *+* _expression_ |call the add service| 2 + 2 |
|<. _expression_ *-* _expression_ |call the sub service| 2 - 2 |
|<. _expression_ *&#42;* _expression_ |call the mult service| 2 * 2 |
|<. _expression_ *&#47;* _expression_ |call the divOp service| 2 / 2 |
|<. _expression_ *<=* _expression_ |call the lessThanEqual service| 2 <= 2 |
|<. _expression_ *>=* _expression_ |call the greaterThanEqual service| 2 >= 2 |
|<. _expression_ *<* _expression_ |call the lessThan service| 1 < 2 |
|<. _expression_ *>* _expression_ |call the greaterThan service| 2 > 1 |
|<. _expression_ *<>* _expression_  |call the differs service| 1 <> 2 |
|<. _expression_ *!=* _expression_  |call the differs service| 1 != 2 |
|<. _expression_ *=* _expression_  |call the equals service| 1 = 1 |
|<. _expression_ *and* _expression_ |call the and service| eClass.interface and eClass.abstact |
|<. _expression_ *or* _expression_ |call the or service| eClass.interface or eClass.abstact |
|<. _expression_ *xor* _expression_ |call the xor service| eClass.interface xor eClass.abstact |
|<. _expression_ *implies* _expression_ |call implies service| eClass.interface implies eClass.abstact |

h4. Structures

|<. *(* _expression_ *)* | parenthesis are used to change priority during evaluation | (2 + 2 ) * 3 |
|<. *if* _expression_ *then* _expression_ *else* _expression_ *endif* | conditional expression | if eClass.abstract then 'blue' else 'red' endif | 
|<. *let* _new_variable_name_ (*:* _type_literal_)? (*,* _new_variable_name_ (*:* _type_literal_)?)* *in* _expression_ | let allows to define variables in order to factorise expression | let container = self.eContainer() in container.eAllContents() |

h4. Literals

|<. *'* _escaped_string_ *'* |you can use java style escape sequence *\u0000* *\x00* *\\* *\'*  *\b* *\t* *\n* ...| 'TODO list:\n\t- walk the dog\n\t- make diner' |
|<. [*0* - *9*]+ |an integer| 100 |
|<. [*0* - *9*]+ *.* [*0* - *9*]+ |a real| 3.14 |
|<. *true* |the boolean value true| true |
|<. *false* |the boolean value false| false |
|<. *null* |the null value| null |
|<. *Sequence{* (_expression_ (*,* _expression_) * ) ? *}* | a sequence defined in extension | Sequence{1, 2, 3, 3} |
|<. *OrderedSet{* (_expression_ (*,* _expression_) * ) ? *}* | an ordered set defined in extension | OrderedSet{1, 2, 3} |
|<. _epackage_name_ *::* _eenum_name_ *::* _eenum_literal_name_ | an EEnumLiteral| art::Color::blue |

h4. Type literals

|<. *String* | the string type | String |
|<. *Integer* | the integer type | Integer |
|<. *Real* | the real type | Real |
|<. *Boolean* | the string type | Boolean |
|<. *Sequence(* _type_litral_ *)* | a sequence type | Sequence(String) |
|<. *OrderedSet(* _type_litral_ *)* | an ordered set type | OrderedSet(String) |
|<. _epackage_name_ *::* _eclassifier_name_ | an eclassifier type | ecore::EPackage |
|<. *{* _epackage_name_ *::* _eclassifier_name_ (*|* _epackage_name_ *::* _eclassifier_name_) * *}* | a set of eclassifiers | {ecore::EPackage &#124; ecore::EClass} |

h2. Migrating from MTL queries

As languages, AQL and MTL are very close yet there are some notable differences:

h3. Implicit variable references

There is no implicit variable reference. With this change, you can easily find out if you are using a feature of an object or a string representation of said object. As a result, instead of using @something@, you must use @self.something@ if you want to access the feature named "something" of the current object or "something" if you want to retrieve the object named something.

In a lambda expression, you must now define the name of the variable used for the iteration in order to easily identify which variable is used by an expression. In Acceleo MTL, you can write @Sequence{self}->collect(eAllContents(uml::Property))@ and Acceleo will use the implicit iterator as a source of the operation eAllContents.

The problem comes when using a lambda like @Sequence{self}->collect(something)@, we can't know if "something" is a feature of "self" or if it is another variable.

Using AQL, you will now have to write either @collect(m | m.eAllContents(uml::Property))@ or @collect(m: uml::Model | eAllContents(uml::Property))@.

h3. Collect and flatten

When a call or a feature acces is done on a collection the result is flattened for the first level. For instance a service returning a collection called on a collection will return a collection of elements and not a collection of collection of elements.

h3.  Type literals & children EPackages

Type literals can't be in the form someEPackage::someSubEPackage::SomeEClass but instead someSubEPackage::SomeEClass should be directly used. Note that the **name of the EPackage is mandatory**. Type literals are handled just like any other type.  

Calls like @self.eAllContents(self.eClass())@ are possible and will return all the children of type compatible with “self”.

Furthermore if you need a type literal as a parameter in your own service, you just have to have a first parameter with the type : @Set<EClass>@. Yes, that’s an important point, any type in AQL is possibly a union of several existing types, hence the collection here. As such the syntax for creating Sets or collections can be used as a substitute for type literals.


h3.  Enumeration literals & children EPackages

Enumeration literal should be prefixed with the name of the containing EPacakge for instance "myPackage::myEnum::value".

h3. Collections

You can only have Sequences or OrderedSets as collections and as such the order of their elements is always deterministic. In Acceleo MTL, you had access to Sets, which are now OrderedSets and Bags, which are now Sequences. Those four kinds of collections were motivated by the fact that Sequence and OrderedSet were ordered contrary to Sets and Bags. On another side, OrderedSets and Sets did not accept any duplicate contrary to Bags and Sequences.

By careful reviewing the use of those collections in various Acceleo generators and Sirius Designers we have quickly found out that the lack of determinism in the order of the collections Sets and Bags was a major issue for our users. As a result, only two collections remain, the Sequence which can contain any kind of element and the OrderedSet which has a similar behavior except that it does not accept duplicates.

Previously in Acceleo MTL, you could transform a literal into a collection by using the operator @->@ on the literal directly. In Acceleo MTL, the collection created was a Bag which is not available anymore. It is recommended to use the extension notation like @Sequence{self}@ or @OrderedSet{self}@. By default in AQL the created collection is an OrderedSet.

h3. Renamed operations

Some operations have been renamed. As such "addAll" and "removeAll" have been renamed "add" and "sub" because those two names are used by AQL in order to provide access to the operator "+" and "-". As a result we can now write in AQL "firstSequence + secondSequence" or "firstSet - secondSet".

h3. Typing

AQL is way smarter than MTL regarding to the types of your expressions. As a result, you can combine expressions using multiple types quite easily. For example, this is a valid AQL expression @self.eContents(uml::Class).add(self.eContents(ecore::EClass)).name@. In Acceleo MTL, we could not use this behavior because Acceleo MTL had to fall back to the concept EObject which does not have a feature "name" while AQL knows that the collection contains objects that are either "uml::Class" or "ecore::EClass" and both of those types have a feature named "name".

h3. null handling

AQL handles null (OclVoid) differently from ocl, a null value will not cause a failure but will be silently handled.
For example, @null.oclIsKindOf(ecore::EClass)@ would have returned true for MTL/OCL, forcing users to use @not self.oclIsUndefined() and self.oclIsKindOf(ecore::EClass)@ instead. This is no longer true in AQL, where "null" doesn't conform to any type, so @null.oclIsKindOf(ecore::EClass)@ will return false. Note that it's still possible to "cast" null in any given classifier. @null.oclAsType(ecore::EClass)@ will not fail at runtime.

Furthermore *oclIsUndefined() does not exist in AQL* and should be replaced by a @...  <> null@ expression.

h2. Migrating from Acceleo2 queries

h3. EClassifier references

All operations referencing a type are now using a type literal with the name of the EPackage and the name of the type instead of a string with the name of the type. As a result, @eObject.eAllContents('EClass')@ would be translated using @eObject.eAllContents('ecore::EClass')@. This allows AQL to now in which EPackage to look for the type and as such, it improves the quality of the validation.

h3. Types and cast

In order to test the type of an EObject, a common pattern in Acceleo 2 was to treat the EObject as a collection and filter said collection on the type desired to see if the size of the collection changed. In AQL, you have access to the operations oclIsTypeOf and oclIsKindOf. You can thus test the type of an EObject with the expression "eObject.oclIsKindOf(ecore::EStructuralFeature)" or "eObject.oclIsTypeOf(ecore::EAttribute)". You can use the operation oclIsKindOf to test if an object has the type of the given parameter or one of its subtype. On the other hand, you can use the operation oclIsTypeOf to test if an object has exactly the type of the given parameter.

Casting in AQL is useless, since AQL is very understandable when it comes to types, it will always tries its best to evaluate your expression.

Since AQL is very close to Acceleo MTL, you can find some additional documentation using the Acceleo equivalence documentation in the "Acceleo documentation":http://help.eclipse.org/mars/index.jsp?topic=%2Forg.eclipse.acceleo.doc%2Fpages%2Freference%2Fmigration.html&cp=5_3_4.

h3. eContainer("TypeName")

In Acceleo2 @self.eContainer("TypeName")@ actually had the behavior of returning self if it was matching the TypeName. As such, when migrating from an eContainer(..) call you should either make sure that this behavior is not needed or use the 
compatibility method provided by AQL : @self.eContainerOrSelf(some::Type)@  



h2. Using AQL programmatically

This section provide information and code snippet. It will help you to integrate AQL in your own tool.

Simple overview of AQL:

!../assets/pics/AQL_overview.png! 

h3. Type validation

For each node of the AST we create a set of possible types as follow:
* for a VarRef we ask the environment for its possible types
* for a FeatureAccess we look up the type of the feature in the registered metamodels
* for a Call we look up the service in the service registry according to the possible types of its parameters (receiver is the first parameter). At this point there is a conversion from EMF to Java. The return type of the service is given by the IService.getType() method. At this point there is a conversion form Java to EMF, one Java type can correspond to more than one EMF EClassifier. If no service can be found we try to find a corresponding EOperation if the receiver is an EObject.

A special type NothingType is used to mark a problem on a given node of the AST. Those NothingTypes are then used to create validation messages. If an AST node has only NothingTypes validation messages will be set as errors for this node, otherwise they are set as warnings.

h3. Completion

The completion rely on the AST production and the type validation.
The identifier fragments preceding (prefix) and following (remaining) the cursor position are removed from the expression to parse. The prefix and remaining are used later to filter the proposals. Many filters can be implemented: filter only on prefix, filter on prefix and remaining, same strategies with support for camel case, ...

Completion on the AST:
* if there is no error node in the AST the completion provide any symbols that can follow an expression ("+", "-", ...).
* if there is an ErrorExpression node in the AST the completion provides anything that can prefix an expression ("not", "-", variable name, type name, ...).
* if there is an ErrorFeatureAccesOrCall node in the AST the completion provides feature and service names corresponding to the receiver possible types. It is also possible to add symbols that follow an expression if the prefix and remaining are already a valid feature or service name for the receiver possible types.
* if there is an ErrorCollectionCall node in the AST the completion provides collection service names. It is also possible to add symbols that follow an expression if the prefix and remaining are already a valid service name.
* if there is an ErrorTypeLiteral node in the AST the completion provides EClassifier, EEnumLiteral names according to the state of the type description.

h3. Creating and setting the environment

To get a fresh environment you can use one of the following snippet:

bc. IQueryEnvironment queryEnvironment = Query.newEnvironmentWithDefaultServices(null);

To get an environment with predefined services.

or

bc. IQueryEnvironment queryEnvironment = Query.newEnvironment(null);

To get an environment with no predefined services. It can be useful to create your own language primitives.

Note that you can also provide a CrossReferenceProvider to define the scope of cross references in your environment. See CrossReferencerToAQL for more details.

You can register new services Class as follow:

bc. ServiceRegistrationResult registrationResult = queryEnvironment.registerServicePackage(MyServices.class);

The registration result contains information about services overrides.

You can also register your EPackages. Only registered EPackages are used to validate and evaluate AQL expression.

bc. queryEnvironment.registerEPackage(MyEPackage.eINSTANCE);

In some cases you might also want to create custom mappings between an EClass and its Class. A basic case is the use of EMap:

bc. queryEnvironment.registerCustomClassMapping(EcorePackage.eINSTANCE.getEStringToStringMapEntry(), EStringToStringMapEntryImpl.class);

By default the EClass is mapped to Map.Entry which is not an EObject. This prevents using services on EObject.

h3. Building an AQL expression

The first step is building your expression from a String to an AST:

bc. QueryBuilderEngine builder = new QueryBuilderEngine(queryEnvironment);
AstResult astResult = builder.build("self.name");

h3. Evaluating an AQL expression

To evaluate an AQL expression you can use the QueryEvaluationEngine

bc. QueryEvaluationEngine engine = new QueryEvaluationEngine(queryEnvironment);
Map<String, Object> variables = Maps.newHashMap();
variables.put("self", EcorePackage.eINSTANCE);
EvaluationResult evaluationResult = engine.eval(astResult, variables);

Here we only use one variable for demonstration purpose.

h3. Validating an AQL expression (optional)

This step is optional for evaluation. You can evaluate an AQL expression without validating it in the first place.

bc. Map<String, Set<IType>> variableTypes = new LinkedHashMap<String, Set<IType>>();
Set<IType> selfTypes = new LinkedHashSet<IType>();
selfTypes.add(new EClassifierType(queryEnvironment, EcorePackage.eINSTANCE.getEPackage()));
variableTypes.put("self", selfTypes);
AstValidator validator = new AstValidator(queryEnvironment, variableTypes);
IValidationResult validationResult = validator.validate(astResult);

h3. Completing an AQL expression

To do this use the QueryCompletionEngine, it will build the query and validate it for you. It will also compute needed prefix and suffix if any:

bc. Map<String, Set<IType>> variableTypes = new LinkedHashMap<String, Set<IType>>();
Set<IType> selfTypes = new LinkedHashSet<IType>();
selfTypes.add(new EClassifierType(queryEnvironment, EcorePackage.eINSTANCE.getEPackage()));
variableTypes.put("self", selfTypes);
QueryCompletionEngine engine = new QueryCompletionEngine(queryEnvironment);
ICompletionResult completionResult = engine.getCompletion("self.", 5, variableTypes);
List<ICompletionProposal> proposals = completionResult.getProposals(new BasicFilter(completionResult));

Here 5 is the offset where the completion should be computed in the given expression.