doc/DSRestAPI/UG/EPTF_CLL_DsRestAPI_UG.adoc

---
Author: Tamás Levente Kiss
Version: 39/198 17-CNL 113 512, Rev. B
Date: 2017-06-12

---
= EPTF Core Library DsRestAPI, User Guide
:author: Tamás Levente Kiss
:revnumber: 39/198 17-CNL 113 512, Rev. B
:revdate: 2017-06-12
:toc:

== About This Document

=== How to Read This Document

This is the User Guide for the DsRestAPI feature of the Ericsson Performance Test Framework (EPTF), Core Library (CLL). CLL is developed for the TTCN-3 <<_1, [1]>> Toolset with TITAN <<_2, [2]>>. This document should be read together with the Function Description of the DsRestAPI feature <<_5, [5]>>. For more information consult the User Guide <<_4, [4]>> and the Function Specification <<_3, [3]>> of CLL.

== System Requirements

In order to use the DsRestAPI feature the system requirements listed in EPTF Core Library User Guide <<_4, [4]>> should be fulfilled.

= DsRestAPI

== Overview

The DsRestAPI feature is developed to provide APIs for the CLL Data Source. It also provides a sub-feature that collects timelines of data elements.

For more information about the Data Source and its purpose, please see <<_8, [8]>> and <<_9, [9]>>.

[[description_of_files_in_this_feature]]
== Description of Files in This Feature

The CLL DsRestAPI feature includes the following files:

* __EPTF_CLL_DsRestAPI_Definitions.ttcn__ - this TTCN-3 module contains common type definitions used by the DsRestAPI Component.
* __EPTF_CLL_DsRestAPI_Functions.ttcn__ - this TTCN-3 module contains the implementation of DsRestAPI functions.
* __EPTF_CLL_DsRestAPI_ExternalFunctions.cc__ - this module contains external functions for DsRestAPI.
* __EPTF_CLL_DsRestAPI_DSServer_Definitions.ttcn__ - this module contains the definitions for the TTCN API of the Data Source.
* __EPTF_CLL_DsRestAPI_DSServer_Functions.ttcn__ - this module contains the implementation of the TTCN API function.
* __EPTF_CLL_DsRestAPI_Timeline_Definitions.ttcn__ - this module contains the definitions for the Timeline sub-feature.
* __EPTF_CLL_DsRestAPI_Timeline_Functions.ttcn__ - this module contains the implementation of the Timeline sub-feature.
* __EPTF_CLL_DsRestAPI_Timeline_ExternalFunctions.cc__ - this module contains external functions for the Timeline sub-feature.
* __EPTF_CLL_DsRestAPI_Filter_Definitions.ttcn__ - this module contains the definitions for the filtering feature.
* __EPTF_CLL_DsRestAPI_Filter_Functions.ttcn__ - this module contains the implementation of the filtering functions.
* __EPTF_CLL_DsRestAPI_Storage_Definitions.ttcn__ - this module contains the definitions for the caching feature.
* __EPTF_CLL_DsRestAPI_Storage_Functions.ttcn__ - this module contains the implementation of the caching functions.
* __EPTF_CLL_DsRestAPI_HTTPServer_Definitions.ttcn__ - this module contains the common type definitions used by the DsRestAPI HTTPServer Component.
* __EPTF_CLL_DsRestAPI_HTTPServer_Functions.ttcn__ - this module contains the implementation of the DsRestAPI HTTP server.
* _DsRestAPIComm.js_ - the JavaScript module that contains the AJAX calls that can be used to communicate with the HTTP server.
* _DsRestAPI.js_ - the JavaScript module that contains the DsRestAPI API functions described in <<_5, [5]>>.
* _DsRestAPI.py_ - the Python module that contains the DsRestAPI API functions described in <<_5, [5]>>.

[[description_of_required_files_from_other_CLL_features]]
== Description of Required Files from Other CLL Features

The DsRestAPI feature depends on the following CLL features:

* `Base`
* `Common`
* `FreeBusyQueue`
* `DataSource`
* `HashMap`
* `Logging`
* `Variable`

== Description of Required External Features Outside CLL

* `TCCUsefulFunctions_CNL113472`
* `IPL4asp_CNL113531`

== Installation

Since `EPTF_CLL_DsRestAPI` is used as a part of the TTCN-3 test environment this requires TTCN-3 Test Executor to be installed before any operation of these functions. For more details on the installation of TTCN-3 Test Executor, see the relevant section of <<_2, [2]>>.

If not noted otherwise in the respective sections, the following are needed to use `EPTF_CLL_DsRestAPI`:

* Copy the files which belong to the CLL features listed in section <<description_of_files_in_this_feature, Description of Files in This Feature>> and <<description_of_required_files_from_other_CLL_features, Description of Required Files from Other CLL Features>> to the directory of the test suite or create symbolic links to them.
* Import the DsRestAPI test or write your own application using EPTF DsRestAPI.
* Create _Makefile_ or modify the existing one. For more details, see the relevant section of <<_2, [2]>>.
* Edit the config file according to your needs, see section <<configuration, Configuration>>.

[[configuration]]
== Configuration

The executable test program behavior is determined through the run-time configuration file. This is a simple text file which contains various sections. The usual suffix of configuration files is _.cfg_. For further information on the configuration file, see <<_2, [2]>>.

The DsRestAPI feature does not define any module parameters.

== General Usage

DsRestAPI provides three main ways to communicate with the Data Source. These three ways are: the TTCN API, the JavaScript API and the Python API.

DsRestAPI provides all three methods of communication, while the standalone DSServer only provides the TTCN API.

To use the DsRestAPI feature, perform the following steps:

1.  Extend your component with the `EPTF_DsRestAPI_CT` component
2.  Call the init function (`f_EPTF_DsRestAPI_init_CT`) of the DsRestAPI to initialize the feature
3.  Call the `f_EPTF_DsRestAPI_start` function to start the DSServer and HTTP server
4.  (Optional) Call `f_EPTF_DsRestAPI_Timeline_start` to start collecting timelines. For more information, see <<_5, [5]>>.
5.  We can now use the three APIs to issue requests:
a.  In TTCN environment use the `f_EPTF_DsRestAPI_DSHandler_processJSONRequest` function to process a request
b.  Use the functions of the JavaScript API to send HTTP requests to the HTTP server
c.  Use the Python API to send HTTP requests to the HTTP server
6.  Call `f_EPTF_Base_cleanup_CT` function before your component terminates

To use the standalone DSServer without starting the HTTP server, fewer steps are required:

1.  Extend your component with the `EPTF_DsRestAPI_DSServer_CT` component
2.  Call the init function (`f_EPTF_DsRestAPI_DSServer_init_CT`) of the DSServer to initialize the feature
3.  Use the `f_EPTF_DsRestAPI_DSHandler_processJSONRequest` function to process a request
4.  Call `f_EPTF_Base_cleanup_CT` function before your component terminates

To use the Timeline as a standalone feature, do the following:

1.  Extend your component with the `EPTF_DsRestAPI_Timeline_CT` component
2.  Call the init function (`f_EPTF_DsRestAPI_Timeline_init_CT`) of the Timeline to initialize the feature
3.  Use the data elements of the DsRestAPI Timeline DataSource Client to collect and access timelines.
4.  Call `f_EPTF_Base_cleanup_CT` function before your component terminates

See <<_5, [5]>> for more details about the API functions.

= Requests and Responses

To communicate with the Data Source, JSON requests need to be sent to the HTTP server. For more detailed information on the format of the `JSON` and its encoding, please see the appropriate parts of <<_5, [5]>>.

== Trees and Branches

Using the _children_ member of the `getData` request, we can create a tree of nested requests.

*Definition*: The _request tree_ is a graph, the nodes of the tree are the requests themselves, and an arc connects request A with request B if B is contained in the children list of A.

This is useful when we want to use the result of one request as a parameter of another one.

*Definition*: We call the requests located on a path from the root to some other request a _branch_ of the request tree.

To refer to the result of the first request (the root) in a branch of the tree in a parameter, use `%Parent0%` in the `paramValue` of the parameter.

Similarly, if we want to refer to the result of the 28th request on the branch, use `%Parent27%` in the `paramValue`.

It is also possible to refer to more than one parent in the `paramValue`. In some cases we might want to have `%Parent0%.%Parent1%` as the `paramValue`.

When the parent is a list, we can also refer to the current index using `%Parent0::idx%`. *Important*: this will only work if the request does not have a filter or a `rangeFilter`.

A `setData` request cannot have children. However, they can be nested in a tree of other `getData` requests.

See section <<examples, Examples>> for examples, and consult the Data Source help and <<_8, [8]>> for the parameters and the possible nesting of the requests.

== Expanding Requests

Suppose we have a request A whose response is a list. What happens to the requests of the children of A?

The answer is, they are processed as many times as the number of elements in the list.

Usually, this is not the intended behavior. We can fine-tune which requests get processed by using the `selection` member of the request A.

The `selection` member of a `getData` request is a list of integers.

When the response to the request is a list and the request has child requests, then the child requests will only be processed on those elements in the list that are specified in the `selection`.

If the selection is an empty list, the child requests will not be processed on any of the elements in the list.

If the selection element is omitted, then the child requests are processed on all elements.

In a `setData` request, if the element is a list and we want to change the value of the third element in the list, we would have `indxsInList`: <<_2, [2]>> as a member of the `setData` request.

For an example, please see section <<examples, Examples>>.

== Filtering Requests

Filters can be used to filter requests and responses based on specific criteria.

=== Range Filter

The `rangeFilter` member of a `getData` request can be used to query only a continuous part of a list.

The offset member of the `rangeFilter` is the index of the first element in the original list, and the count is the maximum number of elements we wish to query.

=== Filter

The `filter` member of a `getData` request can be used to filter a list using conditions.

The `filter` is a rooted tree. If the root evaluates to `_false_` on an element, it is filtered.

If a single element response is filtered, the response will change to an empty string whose `tp` is `_0_` (see <<_5, [5]>> for more details about response types).

If the response is a list, and the filter does not contain any reference to the list elements, then it is treated as a single element response in terms of filtering (the result will be either an empty string with `tp = 0`, or the whole list).

If the response is a list and the filter contains references to its elements, then the filter will be evaluated for every element of the original list. The elements, for which the filter evaluates to `_false_`, will be removed from the result.

See section <<examples, Examples>> for some examples.

=== Evaluating Filters and Selections

Suppose we are processing a request whose response is a list, and it contains a filter with reference to the elements, a `rangeFilter` and selection.

First, the `rangeFilter` is applied. Then we run the filter on all elements. And finally, the selection is applied.

This is why using `%Parent0::idx%` is not advised, since filtering cannot be done in this case.

== Connection Between Request and Response

The response has similar structure to the request.

The response is also a tree and is built according to the rules described previously.

Imagine the response as a graph which is a tree. We can obtain this tree structure from the request tree if we make a copy of certain subtrees. Namely take the node closest to the root whose response is a list. Copy the subtree that belongs to this node as many times as the length of the response list and connect the copies to the parent of the node. Repeat this for all nodes whose corresponding request returns a list.

= Using the APIs

DsRestAPI provides three APIs that can communicate with the Data Source.

== JavaScript

See the description of the `f_EPTF_DsRestAPI_start` function in [5] to correctly configure the HTTP server before using the JavaScript API.

The _DsRestAPIComm.js_ and _DsRestAPI.js_ files have to be included into a web application and a new instance of DsRestAPI has to be created.

Afterwards, the functions described in the JavaScript specific part of <<_5, [5]>> can be used.

== Python

The _DsRestAPI.py_ can be used in two ways.

We can use it as a script with the following command line arguments:

* `-?` print local help (for usage, examples, and types)
* `-h source element` print Data Source help for the given source and element (element or both are optional)
* `-u \http://address:port`: the address of the HTTP server
* `-f file.json`: send the request found in the JSON file
* `-j 'json string'`: send the JSON string as the request

To issue specific `getData` requests, use the following parameters:

* `-source SomeSource`
* `-element SomeElement`
* `-params ParamName1 ParamValue1 ParamName2 ParamValue2`
* `-ptcname SomePTCName`

In addition to these, a `setData` request must contain the following parameters:

* `-content TheContent`
* `-tp SomeInt` (see <<_5, [5]>> for more details)

For further explanations about the arguments, see the Python specific part of <<_5, [5]>>.

We can also import the Python script as a module and use the API class similarly to the JavaScript API.

== TTCN

The full description of the API function can be found in <<_5, [5]>>.

= Warning Messages

NOTE: Besides the below described warning messages, warning and error messages shown in <<_2, [2]>> or those of other used features or product may also appear.

`*DsRestAPI cannot listen on port: port_number*`

This warning indicates that the port assigned to DsRestAPI is already in use.

`*HTTPServer_directory does not exist: dir_name*`

This warning occurs if the given directory for the HTTP server does not exist and it is not the empty string.

`*CustomizableApp directory does not exist: dir_name*`

This warning occurs if the given directory for the customizable app symlink does not exist and it is not the empty string.

`*DsRestAPI API directory does not exist: dir_name*`

This warning occurs if the given directory for the JavaScript API symlink does not exists and it is not the empty string.

`*Symlink creation was unsuccessful for …*`

This warning means that a symlink could not be created.

`*… directory does not exist and could not be created it in …*`

This warning means that a directory creation failed.

`*Symlink cannot be removed: symlink_name*`

This warning appears if a symlink cannot be removed during cleanup.

`*Failed to get absolute path of …*`

This warning means that a relative path could not be resolved.

[[examples]]
= Examples

== Request-Response Pair

See the request-response pair in the figure below:

image:images/Request_response_pair.png[alt]

As we can see, the [red]#EntityGroups# request has a [green]#child request# that uses its value as a parameter.

In addition, the [green]#subrequest# is only processed for the first [red]#entity group# because of the [red]#_selection_# member. This can be observed in the response, where there is a [green]#scenario list# below [red]#DefaultEGrp#, but there is no list below [red]#DefaultEGrp2#.

== A `setData` example

[source]
----
{
    "setData": {
        "source": "ExecCtrl",
        "element": "TcTargetCPSOrWeight",
        "params": [
            {
                "paramName": "EntityGroup",
                "paramValue": "DefaultEGrp"
            },
            {
                "paramName": "Scenario",
                "paramValue": "DefaultSc"
            },
            {
                "paramName": "TrafficCase",
                "paramValue": "DefaultTC1"
            }
        ],
        "content": "20",
        "type": 1
    }
}
----

== A More Complex Example

image:images/amore_complex_example.jpeg[a_more_complex_example]

This is a large request. Only the element names are displayed. In a real world scenario, we might want to control the selection of some requests.

For example, imagine that the first entity groups request does not have a selection, so the response will contain the type and size of every entity group.

The second entity groups request can have a selection which we may control from a user interface, so only the scenarios of the selected entity group will be contained in the response.

== Filtering a Single Element with Value

image:images/Filtering_single_element_with_value.png[alt]

The [red]#filter# will ensure that the `ScTargetCPS` request is only evaluated on weighted scenarios. The response will contain all [green]#scenarios#, and the target cps of weighted ones, and a filtered element response (empty string and `tp=0`) for the other target cps requests.

== Filtering a Single Element with Request

image:images/Filtering_single_element_with_request.png[alt]

The [red]#filter# is now a request with parameters. The result will be the same as in the previous example.

== Filtering a List

image:images/Filtering_a_list.png[alt]

The [red]#filter# is now part of the [green]#scenarios# request and it contains a reference to the list elements: [green]#%Parent1%.# This means that the response scenario list will only contain weighted scenarios.

= Terminology

*Core Library (CLL):* +
It is that part of the TitanSim software that is totally project independent. (i.e., which is not protocol-, or application-dependent). The Core Library is to be supplied and supported by the TCC organization. Any Core Library development is to be funded centrally by Ericsson.

= Abbreviations

CLL:: Core Library

EPTF:: Ericsson Performance Test Framework

TitanSim:: New synonym for the EPTF Framework

TTCN-3:: Testing and Test Control Notation version 3 [1]

GUI:: Graphical user interface

UI:: User interface

API:: Application programming interface

JSON:: JavaScript Object Notation

REST:: Representational State Transfer

AJAX:: Asynchronous JavaScript and XML

= References

[[_1]]
[1] ETSI ES 201 873-1 v3.2.1 (2007-02) +
The Testing and Test Control Notation version 3. +
http://www.etsi.org/deliver/etsi_es/201800_201899/20187301/03.02.01_60/es_20187301v030201p.pdf[Part 1: Core Language]

[[_2]]
[2] User Guide for the TITAN TTCN-3 Test Executor

[[_3]]
[3] EPTF Core Library for TTCN-3 toolset with TITAN, Function Specification

[[_4]]
[4] EPTF Core Library for TTCN-3 toolset with TITAN, User Guide

[[_5]]
[5] EPTF Core Library DsRestAPI, Function Description

[[_6]]
[6] EPTF CLL for TTCN-3 toolset with TITAN +
http://ttcn.ericsson.se/TCC_Releases/Libraries/EPTF_Core_Library_CNL113512/doc/apidoc/html/index.html[Reference Guide]

[[_7]]
[7] Improving TitanSim’s internal communication and user interfaces

[[_8]]
[8] EPTF CLL DataSource, Function Description

[[_9]]
[9] EPTF CLL DataSource, User Guide