﻿openKonsequenz - Architecture of the module "eLogbook@openK"
============================================================
:Author: Frank Dietrich
:Email: frank.dietrich@pta.de
:Date: 2017-07-12
:Revision: 1
:icons:
:source-highlighter: highlightjs
:highlightjs-theme: solarized_dark

This documentation bases on ARC42-Template (v7.0):

*About arc42*

arc42, the Template for documentation of software and system architecture.

By Dr. Gernot Starke, Dr. Peter Hruschka and contributors.

Template Revision: 7.0 EN (based on asciidoc), January 2017

© We acknowledge that this document uses material from the arc 42 architecture template, http://www.arc42.de.
Created by Dr. Peter Hruschka & Dr. Gernot Starke.

<<<

== Introduction and Goals

=== Requirements Overview

The digital logbook (eLogbook@openK) is the main source of information in the central network control unit (CNCU)
for the manual logging of events and operations, which are not recorded automatically in the control system. In
addition to the use for the structured transfer of information during the change of shifts, the company's digital
logbook is to be used as an expanded resource for the work organization in the CNCU and as an information medium for
the well-directed transfer of information from supervisors to employees.

The full requirements of the module eLogbook@openK (in German: Modul Betriebstagebuch) is described in the document

* "Anfragespezifikation Modul Betriebstagebuch" from 15-09-2016.

=== Quality Goals

The eLogbook represents a user module that bases on the architecture platform of openKONSEQUENZ. The main quality
goals of the platform are:

* *Flexibility* The reference platform shall grant that different systems and modules from different vendors/developers can interact and interoperate, and may be exchanged or recombined.
* *Availability* All platform modules that are running on the platform can only be as available as the platform same for user modules that are based on platform modules.
* *Maintainability* (and testability as part of maintainability)  The platform and its platform modules shall be used longer than 15 years.
* *Integration performance* New implemented functionality of oK own modules and external modules shall be included fast / automatically.
* *Security* The platform and its modules need to underly security-by-design

The main quality goals of the user module eLogbook are:

* *Functionality* The user module must fulfil the functional requirements mentioned in the section before
* *Integration performance* The user module must be easy integratable in different production environments. * Modifiability (and testability as part of modifiability) Good documentation (i.e. code and architecture documentation) makes code changes easier and automatic tests facilitate rigorous verification. * Ergonomics The web interface must be realized according to oK-GUI-Styleguide.

The following documents contain the quality goals in detail:

* "Architecture Committee Handbook" v1.2 from 14-09-2016
* "Quality Committee Handbook" v1.1 from 18-08-2016

The project eLogbook@openK bases on the Eclipse Public Lisence 1.0.

=== Stakeholders

.Stakeholders
[options="header,footer"]
|=========================================================
|Role/Name|Contact|Expectations
|Product Owner (represents the Distribution System Operators)|Gordon Pickford, Oliver Tantu|The software must fulfil their functional and nonfunctional Requirements.
|Module Developer|Michel Allessandrini, Jonas Tewolde, Frank Dietrich|All relevant business and technical information must be available for implementing the software.
|External Reviewer (represents the AC/QC)|Martin Jung, Anja Berschneider|The software and the documentation is realized according the Quality and Architecture Handbook of openKONSEQUENZ.
|System Integrator||A documentation for the integration of the module in the DSO specific environments must be available.
|=========================================================

== Architecture Constraints

The main architecture constraints are:

* *Public License* The module must be available under the “Eclipse Public License 1.0”.
* *Availability* The source code of the module must be accessible to any interested person/company. Therefore the project is published at https://projects.eclipse.org/projects/technology.elogbook
* *Standardization* The module must use standardized data structures (CIM) [if available] and the reference platform.

=== Technical Constraints

The following technical constraints are given:

.Technical Contraints
[options="header,footer"]
|========================================================
|Component|Constraints
|Basis components of the reference platform|
- Application Server Tomcat
- JPA EclipseLink
- Database PostgreSQL

|Enterprise Service Bus|
* ESB Talend
* Communication via RESTful Webservices

|Programming Language Frontend
a|* Angular
* Bootstrap
* jQuery
* REST/JSON Interfaces

|GUI design
a|* According to oK-GUI-Styleguide

|Java QA environment|Sonarqube 5.6.6

|Programming Language
a|* Backend: Java 1.8
* Frontend: Angular 4.0.0 (Javascript, Typescript, HTML5, CSS3)

|IDE
a|* Not restricted (Eclipse, IntelliJ, Microsoft Developer Studio, Microsoft Visual Code ...)

|Build system
a|* Backend: Maven
* Frontend: NodeJS + Angular/cli

|Libraries, Frameworks,Components
a|* Used Libraries/Frameworks have to be compatible to the Eclipse Public License
|Architecture Documentation|* According ARC42-Template
|========================================================


=== Technical Dependencies

The following libraries are used:

.Libraries
[options="header,footer"]
|=========================================================
|Name of the library|Version|Artefact-id|Usage|License|Tier
|Node.js|6.10.0 LTS||JavaScript Runtime (server side)|MIT License|Frontend
|npm (in Node.js enthalten)|3.10.10||Node Package Manager (package manager for libraries)|Artistic License 2.0|Frontend
|Angular|4.0.0||UI-Framework|MIT License|Frontend
|Angular Material|2.0.0-beta.2||Angular Material Design Components|MIT License|Frontend
|Bootstrap|3.3.7||CSS-Framework |MIT License|Frontend
|jQuery|3.1.1||JavaScript Bibliothek|MIT License|Frontend
|ng2-daterangepicker|1.0.4||Angular 2 Daterangepicker Component|MIT License|Frontend
|Moment.js|2.16.0||JavaScript library for date and time processing|MIT License|Frontend
|core-js|2.4.1||JavaScript Polyfill for angular support within older browsers|MIT License|Frontend
|rxjs|5.0.1||JavaScript Polyfill for Observables|Apache License 2.0|Frontend
|TS-helpers|1.1.1||Helper library for compiling typescript|MIT License|Frontend
|zone.js|0.7.2||JavaScript Polyfill for asynchronous data binding|MIT License|Frontend
|org.apache.httpcomponents.httpclient|4.5.3
a|
[source,xml]
----
<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.5.3</version>
</dependency>
----
|Backend, Http-Client|Apache 2.0|Backend

|org.json.json|20160810

a|
[source,xml]
----
<dependency>
    <groupId>org.json</groupId>
    <artifactId>json</artifactId>
    <version>20160810</version>
</dependency>
----
|Backend - Json functionality|Json|Backend

|org.jboss.resteasy.resteasy-jaxrs|3.0.21_Final
a|
[source,xml]
----
<dependency>
    <groupId>org.jboss.resteasy</groupId>
    <artifactId>resteasy-jaxrs</artifactId>
    <version>3.0.21.Final</version>
</dependency>
----
|Backend - RestServer|Apache 2.0 / CC0.1.0/ Public|Backend

|org.jboss.resteasy.jaxrs-api|3.0.12.Final
a|
[source,xml]
----
<dependency>
    <groupId>org.jboss.resteasy</groupId>
    <artifactId>jaxrs-api</artifactId>
    <version>3.0.12.Final</version>
</dependency>
----
|Rest-Server|Apache 2.0|Backend

|javax.servlet.servlet-api|3.0.1
a|
[source,xml]
----
<dependency>
    <groupId>javax.servlet</groupId>
    <artifactId>servlet-api</artifactId>
    <version>3.0.1</version>
</dependency>
----
|Backend - Logging Servlet |CDDL GLP 2.0|Backend

|com.google.code.gson.gson|2.8.0
a|
[source,xml]
----
<dependency>
    <groupId>com.google.code.gson</groupId>
    <artifactId>gson</artifactId>
    <version>2.8.0</version>
</dependency>
----
|Backend Json de-serialization|Apache 2.0|Backend

|log4j.log4j|1.2.17
a|
[source,xml]
----
<dependency>
    <groupId>log4j</groupId>
    <artifactId>log4j</artifactId>
    <version>1.2.17</version>
</dependency>
----
|Backend logging|Apache 2.0|Backend

|commons-io|2.5
a|
[source,xml]
----
<dependency>
    <groupId>commons-io</groupId>
    <artifactId>commons-io</artifactId>
    <version>2.5</version>
</dependency>
----
|IO utils|Apache 2.0

|org.eclipse.persistence.eclipselink|2.6.4|Backend
a|
[source,xml]
----
<dependency>
    <groupId>org.eclipse.persistence</groupId>
    <artifactId>eclipselink</artifactId>
    <version>2.6.4</version>
</dependency>
----
|JPA implementation|EDL 1.0 EPL 1.0|Backend

|postgresql.postgresql|9.1-901-1.jdbc4
a|
[source,xml]
----
<dependency>
    <groupId>postgresql</groupId>
    <artifactId>postgresql</artifactId>
    <version>9.1-901-1.jdbc4</version>
</dependency>
----
|DB driver|BSD|Backend

|junit.junit|4.12

a|
[source,xml]
----
<dependency>
    <groupId>junit</groupId>
    <artifactId>junit</artifactId>
    <version>4.12</version>
</dependency>
----
|Unit testing|EPL 1.0|Backend

|org.easymock.easymock|3.4
a|
[source,xml]
----
<dependency>
    <groupId>org.easymock</groupId>
    <artifactId>easymock</artifactId>
    <version>3.4</version>
</dependency>
----
|Unit testing|Apache 2.0|Backend

|org.powermock.powermock-api-easymock|1.6.6
a|
[source,xml]
----
<dependency>
    <groupId>org.powermock</groupId>
    <artifactId>powermock-api-easymock</artifactId>
    <version>1.6.6</version>
</dependency>
----
|Unit testing|Apache 2.0|Backend

|org.jacoco.jacoco-maven-plugin|0.7.9
a|
[source,xml]
----
<dependency>
    <groupId>org.jacoco</groupId>
    <artifactId>jacoco-maven-plugin</artifactId>
    <version>0.7.9</version>
</dependency>
----
|Test coverage|EPL 1.0|Backend
|=========================================================

== System Scope and Context

=== Business Context

The user module eLogbook communicates via the ESB to other modules and systems (see figure 1):

* *Core Module "Auth & Auth"* The eLogbook can only be used by authorized users. Therefore, it is essential to invoke the module “Auth & Auth” for authorization and authentication purposes.
* *Source System "SCADA"* The eLogbook needs information from the system “SCADA”. Therefore, it must provide an interface for receiving the according data.

.System-Context of eLogbook
[options="header,footer"]
image::SystemContext.png[System-Context of eLogbook]

=== Technical Context

The following aspects have to be taken into account for external communication of the module eLogbook:

* As interface-technology RESTful web services are used.
* Each external interface (interfaces between modules or external systems) has to be documented.
* Dependencies of modules to services realized by other modules have to be specified and documented explicitly.
* When CIM is not appropriate (like access management), other standards in their respective domain shall be taken into account first to avoid proprietary and inaccurate interfaces. The interface has also be documented in the overall openKONSEQUENZ interface profile and it should use REST & XML.

The interfaces of the module eLogbook are described in:

* TODO: Bezeichnung und Link einfügen

=== Solution Strategy

The module eLogbook bases on a three-tier architecture:

. *Frontend* - The GUI is implemented as a web-frontend with rich-client functionalities.
. *Backend* - The business functionalities are implemented in the backend tier. It provides the business functions via RESTful Webservices.
. *Database* - The database stores all module specific data.

== Building Block View

=== Whitebox Overall System

The module eLogbook contains two components (see figure 2):

. *UI* - Represents the graphical user interface and consumes the services from the Business logic component via RESTful webservices.
. *Business Logic* - Realizes the business functionality and the data storage of the module.

.Module components
[options="header,footer"]
image::ModuleComponents.png[Module components]

Bases on the abstract concept mentioned above, the following figure shows the concrete realization of the main components.
The mapping from the abstract to the concrete view is as follows:

. UI
 - elogbookFE-SPA
. Business Logic
 - elogbook.war (Rest-Service)
 - elogbook-DB (in figure called: Betriebstagebuch DatenbankQU)

.Distribution of components
[options="header,footer"]
image::DistributionOfComponents1.png[DistributionComp]

==== elogbook.war (Backend tier)

This component implements the business functionality of the digital logbook. And it provides services, that the
elogbookFE – SPA can use the functions in the frontend.

The elogbook.war runs on the Apache Tomcat in the DSO specific environments (in the figures shown as openKONSEQUENZ - LAN)

==== elogbook-DB (Database tier)

This component stores the data of the digital logbook. It provides an interface to the elogbook.war to create or
change data in the database.

The elogbook-DB runs on a Postgre DBMS.

==== eLogbook-API

The eLogbook needs information from the system "SCADA". Therefore, it must provide an interface for receiving the according data, see figure 1.

=== Level 2

==== elogbookFE – SPA (Frontend tier)

The frontend component implements the concept of a single-page application (SPA). The framework used is Angular2. It divides
the elogbookFE into three layers:

. *Components* - The components (Pages, Lists, Dialogs, Common Comp.) represent the presentation layer and the control layer. A component contains the control logic (.ts-file), an HTML-fragment as presentation description (.html-file) and a style definition (.css-file).
. *Services* - The service component communicates with the interfaces of the backend via HTTP requests by using the model component.
. *Model* - The model corresponds to the View-Model of the backend tier.

.Frontend tier
[options="header,footer"]
image::FrontendTier.png[]

==== elogbook.war (Backend tier)

The backend tier contains five components which can be summarized in three layers:

. *Presentation layer* - Represented by
 .. REST-Srv
 .. View Model
. *Controller layer* - Represented by
 ..	Controller
.	*Model layer* - Represented by
 ..	DAO
 ..	Model

.Backend tier
[options="header,footer"]
image::BackendTier.png[]

==== elogbook-DB (Database tier)

The elogbook-DB is realized as a relational database system.

.Database tier
[options="header,footer"]
image::DatabaseTier.png[]


== Runtime View

=== Login / Authentication

There is no login page, since the openK-Portal-Application is responsible for Authentication and
the whole SSO (single sign on) process.
Therefore the elogbook application has to be started by providing a valid authentication token.
This token is a JWT (JSON Web Token).

.eLogbook application is called by the *portal* application. The User is already logged in
[plantuml]
....
actor User
participant PortalFrontend
participant PortalBackend
participant eLogbookFrontend
entity eLogbookFEStorage
participant eLogbookBackend


User->PortalFrontend: Start eLogbook(JWT)
PortalFrontend->eLogbookFrontend: nav. to frontend-URL with JWT
eLogbookFrontend->eLogbookFEStorage: Extract JWT and store token in session
... some delay ...
eLogbookFrontend->eLogbookBackend: Call any secured service with JWT
group Call secured service

    eLogbookBackend->PortalBackend: "/checkAut(JWT)"
    group Authorization succeeded
        eLogbookBackend->eLogbookBackend: run service
        eLogbookBackend->eLogbookFrontend: return service result
    end
    group Authorization failed
        eLogbookBackend->eLogbookFrontend: return HTTP Code 404
    end
end
....


=== Shift Change

The precondition for performing a shift change is, that one or more users already have
the responsibility for a set of grid territory/branch combinations.

In the following example the user 'Max' wants to quit his shift and therefore needs to
transfer his responsibilities to other users (in our case to 'Pete' and 'Hugo').
He presses the Button "Schicht übergeben" (-> Transfer Shift) on the main overview page.

First the frontend reads all users from the backend. After that the backend reads all current
responsibilities of Max (The backend determines "Max" from the given session info) and returns them.

On the dialog "Schichübergabe" (->Shift change) Max assigns all of his current responsibilities to
'Hugo' and 'Pete'. Both users are now *planned* for responsibilities.


.Max initiates a shift change
[plantuml]
....
actor Max
participant Frontend
participant Backend
entity Database

Max->Frontend: press "Schicht übergeben"


Frontend->Backend: Service "users"
Backend-->Frontend: all users

Frontend->Backend: Service "currentResponsibilities" [SessionInfo]
Backend->Database: read current Resp. for [SessionInfo.Max]
Database-->Backend
Backend-->Frontend:

Frontend-->Max: Show dialog "Schichtübergabe"

Max->Frontend: Select users
Frontend-->Max

Max->Frontend: assign Pete and Hugo
Frontend-->Max

Max->Frontend: press "übergeben"
Frontend->Backend: Service "planResponsibilities"
Backend->Database: store 'Pete' and 'Hugo' as planned responsibilities
Database-->Backend:
Backend-->Frontend:
Frontend-->Max:
....

After the dialog "Schichtübergabe" 'Pete' and 'Hugo' are stored as
*planned responsibilies* for different combinations of grid terr./branch.

.Pete logs in
[plantuml]
....
actor Pete
participant Frontend
participant Backend
entity Database

Pete->Frontend: Login
Frontend->Backend: plannedResponsibilities [SessionInfo]
Backend->Database: read planned resps for [SessionInfo.Pete]

Database-->Backend: result
Backend-->Frontend: result
Frontend->Frontend: check if result not empty
Frontend-->Pete: (then) Show "Schichtübergabe Protokoll"

Pete->Frontend: Select/Deselect responsibilities
Frontend-->Pete:

Pete->Frontend: press "Übernehmen"
Frontend->Backend: Service "confirmResponsibilities"
Backend->Database: Switch the selected resps. from "planned" to "current"
Database-->Backend:
Backend->Database: Store as historical responsibility
Database-->Backend:
Backend-->Frontend:
Frontend-->Pete:
....

When 'Pete' presses "Übernehmen" (->"Take Responsibilities") all the selected
responsibilities for 'Pete' (regarding grid territory and branch) are switched
from 'planned' to 'current' in a single transaction. That means, that 'Pete'
is no longer planned for a set of responsibilities. From now on he is responsible
for them. In the same transaction the data of this process is stored as the historical data
(this is described in another chapter).

When 'Hugo' logs in, it is the same process as for 'Pete'.

If 'Hugo' or 'Pete' decide to deselect a responsibility they are planned for,
then that one will be kept with 'Max' as responsible person. 'Max' will then
need to plan it for another person.


== Deployment View

The elogbook application consists of 3 part regarding the deployment.

. Frontend: "elogbookFE"-Directory
. Backend: "elogbook.war"
. Database: The scripts are found in the "db" folder within the backend sources.

=== Deployment of the application components

==== Deployment of the frontend

The Frontend SPA is built as a folder, that contains all the required files for
the application. When deploying the frontend application, the content of the "dist"-folder
within the Frontend development-directory has to be copied into the
target-directory of the apache-tomcat:

 <Apache-Tomcat>/webapps/elogbookFE

If the target folder does not exist, it has to be created manually.

==== Deployment of the backend
The backend REST-services are built in form of a "WAR"-file (to be found
in the "target"-directory of the MAVEN-directory structure).
For deployment, this file has to be copied to the directory

 <Apache-Tomcat>/webapps

==== Deployment of the database
Currently there is no automatic mechanism for distributing structural
or content related changes to the database.
DB related changes are deployed using database scripts directly in suitable
DB management applications like *pgAdminIII* or directly in the *postgres-console*.
These script can be found in the backend project in the directory

 <elogbook backend project directory>/db

CAUTION: Consider the order of the scripts!

==== Configuration of the system

===== DB based configuration

. Grid territories - The grid-territories have to be configured in the DB-Table *ref_grid_territory*

===== Configuration of the webserver

There exists the file *context.xml* in the "conf" subdirectory (*<TOMCAT>/conf*) of the target apache tomcat installation.
Add the following parameter and resource in order to access the correct backend configuration and to
gain access to the database:

.context.xml
[source,xml]
----
[...]
    <!-- Uncomment this to disable session persistence across Tomcat restarts -->

    <!--Manager pathname=""/>


    <Parameter name="environment" override="false" value="Development"/-->

    <Parameter name="OK_ELOGBOOK_ENVIRONMENT" override="false" value="Production"/>

    <Resource name="jdbc/okBetriebstagebuchDS"
              auth="Container"
              type="javax.sql.DataSource"
              driverClassName="org.postgresql.Driver"
              url="jdbc:postgresql://{dbserver}:5432/{dbname}"
              username="{dbuser}"
              password="{dbpassword}"/>


    <!-- Uncomment this to enable Comet connection tacking (provides events
         on session expiration as well as webapp lifecycle) -->
[...]
----

*{dbserver}*, *{dbname}*, *{dbuser}* and *{dbpassword}* need to be replaced by the actual values.

CAUTION: The postgres database driver (postgresql-42.0.0.jar) referenced in the *context.xml* needs to be copied from the lib folder within the
backend source files to the *<TOMCAT>/lib* folder of the target tomcat installation.

===== Configuration of the backend

After the backend war file has been deployed and unpacked inside of the *<TOMCAT>/webapps* folder there are different
 backend config files to be found in the folder *<TOMCAT>/webapps/elogbook/WEB-INF/classes*

* backendConfigCustom.json
* backendConfigDevLocal.json
* backendConfigDevServer.json
* backendConfigProduction.json

The active configuration is chosen by parameter *OK_ELOGBOOK_ENVIRONMENT* (see context.xml above).
Possible values are:

* *Custom* (for backendConfigCustom.json)
* *DevLocal* (for backendConfigDevLocal.json)
* *DevServer* (for backendConfigDevServer.json)
* *Production* (for backendConfigProduction.json)

After choosing an environment the corresponding json file has to be configured:

.backendConfigXXX.json
[source,json]
----
{
  "portalBaseURL" : "http://{hostname}:{port}/portal/rest/beservice",
  "fileRowToRead": "9",
  "importFilesFolderPath": "/home/btbservice/importFiles",
  "activePersistencyUnit": "betriebstagebuchORA"
}
----
* *portalBaseURL* - The portal base url should point the the address where the portal backend (the *Auth&Auth*-Server)
                    can be accessed. This is important for the authorization of the backend services.
* *fileRowToRead* - Defines the line number inside the import-file of the relevant text line.
* *importFilesFolderPath* - Defines the exchange directory for the import functionality
* *activePersistencyUnit* - Defines the active part of the file persistenc.xml. Choose *"betriebstagebuchORA"* for the
                    usage of OracleDB and *"betriebstagebuch"* to use PostgreSQL.


=== CI- and CD-Components

==== GIT-Repository

<TODO>

==== Hudson

<TODO>

==== Sonar

<TODO>

==== Code-Coverage of the Frontend - Karma-Istanbul-Output

<TODO>

=== Continuous Deployment

The continuous deployment is realized on two platforms:

 * the development platform (Dev-Environment)
 * the quality platform (Q-Environment)

The automatic deployment on both of the environments is
directly linked to the branches on the GIT-repositories:

. "SNAPSHOT" or "DEVELOP"
. "MASTER" or "TRUNC"

The running development is exclusively made on the Snapshot-Branch. Every time
a developer checks in (pushes) code to the repository, an automatic build
starts on the hudson ci-server. If the Snapshot-build is successful, then
outcome of that build is directly deployed on the Dev-Environment.

At the end of a scrum sprint or when a big userstory is realized, all
the code changes are merged from the *Snapshot*-Branch to the *Trunc*.
This automatically triggers the build and the deployment on the
Q-Environment.

CAUTION: Changes on the database are not deployed automatically!


== Design Decisions

All architecture decisions based on the Architecture Committee Handbook. There are no deviations.
 

== Quality Requirements

TODO: Muss noch beschrieben werden

=== Quality Tree

TODO: Muss noch beschrieben werden

=== Quality Scenarios

TODO: Muss noch beschrieben werden


== Risks and Technical Debts

(Muss noch beschrieben werden)

<<<

== Glossary

.Technical Contraints
[options="header,footer"]
|========================================================
|Short|Long|German|Description
|AC|Architecture Committee|Architektur-Komittee|Gives Framework and Constraints according to architecture for oK projects.
|CNCU|Central Network Control Unit||
|DAO|Data Access Objects||
|DSO|Distribution System Operator|Verteilnetz-betreiber (VNB)|Manages the distribution network for energy, gas or water.
|EPL|Eclipse Public License||Underlying license model for Eclipse projects like elogbook@openK
|ESB|Enterprise Service Bus||Central instance for exchange of data to overcome point-to-point connections.
|oK|openKONSEQUENZ|openKONSEQUENZ|Name of the consortium of DSOs
|QC|Quality Committee|Qualitätskomitee|Gives framework and constraints according to quality for oK projects.
|SCADA|Supervisory Control and Data Acquisition|Netzleitsystem|System, that allows DSOs view/control actual parameters of their power grid.
|========================================================