////
 *******************************************************************************
 * Copyright (c) 2019 Contributors to the Eclipse Foundation
 *
 * See the NOTICE file(s) distributed with this work for additional
 * information regarding copyright ownership.
 *
 * This program and the accompanying materials are made available under the
 * terms of the Eclipse Public License v. 2.0 which is available at
 * http://www.eclipse.org/legal/epl-2.0.
 *
 * SPDX-License-Identifier: EPL-2.0
 *******************************************************************************
////

= openKonsequenz - Architecture of the module 'Grid Failure Information'
Michel Alessandrini <michel.alessandrini@pta.de>; Frank Dietrich <frank.dietrich@pta.de>; Simon Reis <simon.reis@pta.de>
:revnumber: 0.2
:revdate: 03.07.2020
:revremark:  Inital Draft
<<<

Based on the ARC42-Template (v7.0):

* Version: {revnumber}
* Date: {revdate}
* Status: {revremark}

<<<

:icons:
:source-highlighter: highlightjs
:highlightjs-theme: solarized_dark

:lang: en
:encoding: utf-8

:imagesdir: ../img
:iconsdir: ../img/icons

:toc:
:toclevels: 4
:toc-title: Table of Contents
:toc-placement!:
:sectanchors:
:numbered:

toc::[]

<<<

== Introduction and Goals

=== Requirements Overview
The module 'Grid Failure Information' (in German: Modul 'Störungsinformationstool')
informs employees and customers about failures and planned interruptions of supply.
The module realizes the whole process from the creation of a failure information,
via the publication until the closure of the failure information.

It supports the user by the acquisition of data and leads the user through
the workflow for processing a failure information.

The module has many interfaces to other modules and external systems.

The full requirements of the module 'Grid Failure Information'  is described in the document

* "Anforderungsspezifikation Modul Störungsinformationstool" version 1.3 / 31-07-2019.


=== Quality Goals
The module 'Grid Failure Information' represents a user module that is based on the architecture platform of openKONSEQUENZ. The main quality
goals of the platform are:

* *Flexibility* The reference platform shall grant that different systems and modules from different vendors/developers can interact and interoperate, and may be exchanged or recombined.
* *Availability* All platform modules that are running on the platform can only be as available as the platform same for user modules that are based on platform modules.
* *Maintainability* (and testability as part of maintainability)  The platform and its platform modules shall be used longer than 15 years.
* *Integration performance* New implemented functionality of oK own modules and external modules shall be included fast / automatically.
* *Security* The platform and its modules need to underly security-by-design

The main quality goals of the core module Contact Base Data are:

* *Functionality* The core module must fulfil the functional requirements mentioned in the section before
* *Ergonomics* The web interface must be realized according to oK-GUI-Styleguide.
* *Good documentation* (i.e. code and architecture documentation) makes code changes easier and automatic
tests facilitate rigorous verification.
* *Modifiability* (and testability as part of modifiability)
* *Integration performance* The core module must be easy integratable in different production environments.

The following documents contain the quality goals in detail:

* Architecture Committee Handbook v1.6.0 from 10-07-2019
* Quality Committee Handbook v2.0.1 from 15-10-2018

The architecture is based on the AC-Handbook. The quality demands are described in the QC-Handbook.
Both specifications were fully complied with in the project, so that a high quality is given.

The code quality regarding static code analysis and unit test code coverage on the backend and fronend sides
are ensured by the use of sonarqube. The rule set and the qualtity gate are defined by the default, the
so called "sonar way".

The module 'Grid Failure Information' is part of the Eclipse project 'Eclipse openK User Modules'.
This project bases on the Eclipse Public Licence 2.0.

=== Stakeholders

.Stakeholders
[options="header"]
|=========================================================
|Role/Name|Contact|Expectations
|Product Owner (represents the Distribution System Operators)|Maike Salbeck, Carsten Otten, Benedikt Herget, Rainer Fuhrmann|The software must fulfil their functional and nonfunctional requirements.
|Module Developer|Michel Alessandrini, Jonas Tewolde, Frank Dietrich|All relevant business and technical information must be available for implementing the software.
|External Reviewer (represents the AC/QC)|n.n.|The software and the documentation is realized according to the Quality and Architecture Handbook of openKONSEQUENZ.
|External Reviewer (represents the Eclipse-Requirements)|n.n.|The software is  licensed under the EPL 2.0. It must be validated that all requirements are fulfilled.
|System Integrator|n.n.|A documentation for the integration of the module in the DSO specific environments must be available.
|=========================================================

<<<

== Architecture Constraints

The main architecture constraints are:

* *Public License* The module must be available under the “Eclipse Public License 2.0”.
* *Standardization* The module must use the reference platform.
* *Availability* The source code of the module must be accessible to any interested person/company.

Therefore the project is published under the following repositories:

* https://git.eclipse.org/r/openk-usermodules/org.eclipse.openk-usermodules.gridFailureInformation.backend
* https://git.eclipse.org/r/openk-usermodules/org.eclipse.openk-usermodules.gridFailureInformation.frontend
* https://git.eclipse.org/r/openk-usermodules/org.eclipse.openk-usermodules.gridFailureInformation.documentation


=== Technical Constraints

The following technical constraints are given:

.Technical Contraints
[options="header"]
|========================================================
|Component|Constraints
|Base components of the reference platform
a|* Application Server Tomcat
* JPA EclipseLink
* Database PostgreSQL

|Programming language frontend
a|* Angular
* Bootstrap
* jQuery
* REST/JSON Interfaces

|GUI design
a|* According to oK-GUI-Styleguide

|Java QA environment
a| * Sonarqube 5.6.6

|Programming language
a|* Backend: Java 1.8
* Frontend: Angular 7+ (Javascript, Typescript, HTML5, CSS3)

|IDE
a|* Not restricted (Eclipse, IntelliJ, Microsoft Developer Studio, Microsoft Visual Code ...)

|Build system
a|* Backend: Maven
* Frontend: NodeJS + Angular/cli

|Libraries, frameworks, components
a|* Used Libraries/Frameworks have to be compatible to the Eclipse Public License

|Architecture Documentation
a|* According ARC42-Template
|========================================================


=== Technical Dependencies

==== Modules
The following modules are required to use the 'Grid Failure Information':

.Modules
[options="header"]
|=========================================================
|Name of the module|Purpose|Status of the module|status
|'Auth&Auth'|Authentification and Authorization|available|required
|'Contact base data'|Contact Base Data|available|required
|'eLogbook'|Log Book|available|optional
|'Planned Grid Measures'|Planned Grid Measures|available|optional
|'CIM-Cache'|Interface to the SCADA system|available|optional
|=========================================================


==== Libraries

The following libraries are used:
TODO: Am Ende die Bibliotheken aktualisieren

.Libraries
[options="header"]
|=========================================================
|Name of the library|Version|Artefact-id|Usage|License|Tier


|org.springframework.boot.spring-boot-starter-parent|2.2.1.RELEASE
a|
||Apache License 2.0|Backend

|org.springframework.boot.spring-boot-starter-data-jpa|2.2.1.RELEASE
a|
||Apache License 2.0|Backend

|org.springframework.boot.spring-boot-starter-oauth2-client|2.2.1.RELEASE
a|
||Apache License 2.0|Backend

|org.springframework.boot.spring-boot-starter-security|2.2.1.RELEASE
a|
||Apache License 2.0|Backend

|org.springframework.boot.spring-boot-starter-web|2.2.1.RELEASE
a|
||Apache License 2.0|Backend

|org.flywaydb.flyway-core|6.0.8
a|
||Apache License 2.0|Backend

|org.springframework.cloud.spring-cloud-starter-openfeign|2.2.0.RELEASE
a|
||Apache License 2.0|Backend

|org.springframework.cloud.spring-cloud-starter-netflix-ribbon|2.2.0.RELEASE
a|
||Apache License 2.0|Backend

|org.keycloak.keycloak-core|3.4.2_Final
a|
||Apache License 2.0|Backend

|org.postgresql.postgresql|42.2.8
a|
||New BSD License|Backend

|org.projectlombok.lombok|1.18.10
a|
||MIT|Backend

|org.mapstruct.mapstruct-processor|1.2.0.Final
a|
||Apache License 2.0|Backend

|io.jsonwebtoken.jjwt|0.9.1
a|
||Apache License 2.0|Backend

|io.springfox.springfox-swagger2|2.9.2
a|
||Apache License 2.0|Backend

|io.springfox.springfox-swagger-ui|2.9.2
a|
||Apache License 2.0|Backend

|org.springframework.boot.spring-boot-starter-test|2.2.1.RELEASE
a|
||Apache License 2.0|Backend

|org.springframework.security.spring-security-test|5.2.1.RELEASE
a|
||Apache License 2.0|Backend

|org.powermock.powermock-reflect|2.0.0
a|
||Apache License 2.0|Backend

|com.h2database.h2|1.4.200
a|
||EPL|Backend

|org.springframework.cloud.spring-cloud-dependencies|Hoxton.RELEASE
a|
||Apache License 2.0|Backend

|org.springframework.boot.spring-boot-maven-plugin|2.2.1.RELEASE
a|
||Apache License 2.0|Backend

|org.jacoco.jacoco-maven-plugin|0.7.9
a|
||EPL 2.0|Backend

|org.sonarsource.scanner.maven.sonar-maven-plugin|3.2
a|
||LGPL 3.0|Backend

|org.asciidoctor.asciidoctor-maven-plugin|1.5.3
a|
||Apache License 2.0|Backend


|org.jruby.jruby-complete|9.0.0.0
a|
||EPL 2.0|Backend

|org.asciidoctor.asciidoctorj-pdf|1.5.0-alpha.11
a|
||Apache 2.0|Backend


|org.asciidoctor.asciidoctorj|1.5.4
a|
||Apache 2.0|Backend


|org.asciidoctor.asciidoctorj-pdf|1.5.0-alpha.11
a|
||Apache 2.0|Backend


|org.asciidoctor.asciidoctorj-diagram|1.5.4.1
a|
||Apache 2.0|Backend


|Angular Font Awesome|3.1.2
a|
||MIT License|Frontend


|@auth0/angular-jwt|3.0.1
a|
||MIT License|Frontend


|font-awesome|4.7.0
a|
||MIT License|Frontend


|@ngrx/core|1.2.0
a|
||MIT License|Frontend


|@ngrx/effects|8-2-0
a|
||MIT License|Frontend


|@ngrx/store|8.3.0
a|
||MIT License|Frontend


|@ngrx/store-devtools|8.2.0
a|
||MIT License|Frontend


|@ngx-translate/core|11.0.1
a|
||MIT License|Frontend


|@ngx-translate/http-loader|4.0.0
a|
||MIT License|Frontend


|ag-grid-angular|21.2.1
a|
||MIT License|Frontend


|ag-grid-community|21.2.1
a|
||MIT License|Frontend


|angular2-notifications|2.0.0
a|
||MIT License|Frontend


|bootstrap|4.4.1
a|
||MIT License|Frontend


|jquery|3.4.1
a|
||MIT License|Frontend


|classlist.js|1.1.20150312
a|
||MIT License|Frontend


|core-js|3.2.1
a|
||MIT License|Frontend


|moment|2.24.0
a|
||MIT License|Frontend


|ng2-popover|0.0.14
a|
||MIT License|Frontend


|ngrx-forms|5.2.1
a|
||MIT License|Frontend


|npm-install-peers|1.2.1
a|
||MIT License|Frontend


|reselect|4.0.0
a|
||MIT License|Frontend


|rxjs|6.5.3
a|
||MIT License|Frontend


|rxjs-compat|6.5.4
a|
||MIT License|Frontend


|ts-helpers|1.1.2
a|
||MIT License|Frontend


|tslib|1.10.0
a|
||MIT License|Frontend


|web-animations-js|2.3.2
a|
||MIT License|Frontend


|zone.js|0.10.1
a|
||MIT License|Frontend


|@swimlane/ngx-datatable|15.0.2
a|
||MIT License|Frontend


|puppeteer|2.0.0
a|
||MIT License|Frontend


|ngx-toastr|11.2.1
a|
||MIT License|Frontend

|popper.js|1.16.0
a|
||MIT License|Frontend

|@ng-bootstrap|5.1.5
a|
||MIT License|Frontend


|=========================================================

<<<

== System Scope and Context

=== Business Context

The core module 'Grid Failure Information' communicates via Restful Webservices with the follwowing modules:

* *Core Module 'Auth & Auth'* The 'Grid Failure Information' can only be used by authorized users.
Therefore, it is essential to invoke the module 'Auth & Auth' for authorization and authentication
purposes.

* *Core Module 'Contact base data'* The user management can only by done by the module contact base data.
Therefore, it is essential to user the module ''Contact base data' for managing all users and customers.

* *Core Module 'CIM-Cache'* The 'Grid Failure Information' can only communicate via the CIM-Cachewith the SCADA system .
Therefore, it is essential to invoke the module 'CIM-Cache' for getting information from the SCADA system.

* *User Module 'eLogbook'* The 'Grid Failure Information' can import failure information from the module 'eLogbook'.

* *User Module 'Planned Grid Measures'* The 'Grid Failure Information' can import failure information from the module 'Planned Grid Measures'.


=== Technical Context

The following aspects have to be taken into account for external communication of the module 'Grid Failure Information':

* RESTful web services are used as interface-technology.
* Each external interface (interfaces between modules or external systems) has to be documented.
* Dependencies of modules to services realized by other modules have to be specified and documented explicitly.

The interfaces of the module 'Grid Failure Information' are described in the interface documentation.
TODO: Link zur interface documentation einfügen

=== Solution Strategy

The module 'Grid Failure Information' bases on a small microservice architecture, including
an asynchronous messaging system.

<<<

== Building Block View

=== Whitebox Overall System

The module 'grid failure information' contains several components:

. *SIT-Web-FE* - This component (SPA with Angular) provides two HTML pages: a table and a map
with failure information. This component is in the DMZ and can be called up from the Internet.
. *SIT-Web-Cache-Service* - The SIT-Web-Cache-Service is a service which receives the published
grid failures in the form of a JSON from the SIT-BE and makes them available to the SIT-Web-FE
via a ReST interface. This component is implemented with Angular / Typescript. Because this
component is only equipped with freely accessible data, no security is necessary here. The
SIT-Web-Cache-Service, like the SIT-Web-FE, is situated in the DMZ.
. *SIT-FE* -
This component (SPA with Angular) provides the user interface for the failure information application. The SIT-FE
receives its authentication when it is called from the PortalFE in the form of a JWT.
. *SIT_BE* - The SIT-BE (Java Spring Boot Microservice) provides all CRUD services in the form of
ReST services that the frontend requires. The SIT-BE is the only component that has
access to the database. Every call is authorized against the PortalBE. This microservice also includes
the JobManager subcomponent.
. *JobManager* - The various imports and exports are controlled via the JobManager. The JobManager
knows all available import and export jobs and can control them via the internal message bus
(RabbitMQ internal). Because the communication between the JobManager and the jobs takes place
via the message bus, the JM does not need to know their configuration and URLs. For this, the JM must
ensure synchronization (incl. timeout behavior) for asynchronous message communication.
. *Import- and export jobs* - All jobs provide a uniform (MessageBus) interface to start a job,
return the result of a job, or to provide information about the respective job (name, version, timeout, status, etc.).
The job manager cyclically requests all configured jobs to send current status information to it.
The interfaces of the individual jobs "outside" can be very different (file system, Internet, message queue, etc.)

[#architecture-overview]
.Architecture of the grid failure information tool
[options="header"]
image::architectureSIT.png[]


=== Level 2

==== GridFailureInformationFE (frontend tier)

(Git-Project: _gridFailureInformation.frontend/grid-failure-information-app_, *SIT-FE*)

The frontend component implements the concept of a single-page application (SPA). The framework used is Angular5.

It divides the gridFailureInformationFE into three layers:

. *Components* - The components (pages, lists, dialogs, common comp.) represent the presentation layer and the control layer. A component contains the control logic (.ts-file), an HTML-fragment as presentation description (.html-file) and a style definition (.css-file).
. *Services* - The service component communicates with the interfaces of the backend via HTTP requests by using the model component.
. *Model* - The model corresponds to the view-model of the backend tier.

.Frontend tier
[options="header"]
[plantuml]
----

node gridFailureInformation_Frontend  {
    component Model


    node Components {
        component "Pages"
        component Lists
        component "Common Components"
    }

    component Services

    Components --> Services
    Components --> Model
    Services --> Model
}

node "Grid Failure Information Backend (simplified)" {
    component RestService
    component ViewModel_API__DTO
}

Services .. RestService
Model .. ViewModel_API__DTO
RestService --> ViewModel_API__DTO
----

==== Grid Failure Frontend Map- and Table-app

(Git-Project: _gridFailureInformation.frontend/grid-failure-information-map-app_,
_gridFailureInformation.frontend/grid-failure-information-table-app_ and grid-failure-information-web-comp, *SIT-WEB-FE*)

Two web components representing the embeddable SIT-Web-Frontend for the internet


==== Grid Failure Web-Cache Service

(Git-Project: _gridFailureInformation.frontend/grid-failure-information-web-cache_, *SIT-Web-Cache-Service*)

...as described above...

==== grid-failure-information (backend tier)

(Git-Project: _gridFailureInformation.backend/gfsBackendService_, *SIT-BE*)

The backend tier contains five components which can be summarized in three layers:

. *Presentation layer* - Represented by
.. REST-Srv
.. View model/DTO
. *Controller layer* - Represented by
..	Controller
..  Service
.	*Model layer* - Represented by
..  Repository
..	Model


.Backend tier
[options="header"]
[plantuml]
----

node "Grid Failure Information Backend"  {
    component Model


    component RestService

    component ViewModel_DTO

    component Controller

    component Service

    component Repository

    RestService --> ViewModel_DTO
    RestService --> Controller
    Controller --> Service
    Service --> Repository
    Repository --> Model
     }

node DBMS {
    component GridFailureInformationDB
}

Repository --> GridFailureInformationDB
----

==== GridFailureData-DB (database tier)


The GridFailure-DB is realized as a relational database system.

.Database tier
[options="header"]
[plantuml]
----

node DBMS {
    component GridFailureDB
}
----

==== AdressImport

(Git-Project: _gridFailureInformation.backend/addressImport_)

Separate component that manages the import of adresses and stations for the grid failure information system.


==== MailExport

(Git-Project: _gridFailureInformation.backend/mailExport_, *ExportJobs*)

Microservice that is part of the export jobs in the diagram above. This service is responsible
for the distribution and sending of e-mails.

==== SAMO-Interface

(Git project: _gridFailureInformation.backend/SAMO-Interface_, *ImportJobs*))

Microservice that is part of the import jobs in the diagram above. The service manages the
import of messages from the external SAMO system. It is a file based import, that needs
the correct configuration of an import file directory.

==== StoerungsauskunftInterface

(Git-Project: _gridFailureInformation.backend/stoerungsauskunftinterface, *ImportJobs* & *ExportJobs*)

Stoerungsauskunftinterace is an Microservice that provides a bi-directional interface to the web site "Störungsauskunft.de".
It is able to import and to export failure informations from/to Störungsauskunft.de.

==== Test importer

(Git-Project: _gridFailureInformation.backend/testImportGridFailures, *ImportJobs* )

Testimport microservice that can be used to test the import mechanism and the correctness of the rabbitMQ configuration.


<<<

== Runtime view

=== Login / authentication


There is no login page, since the openK-Portal-Application is responsible for authentication and
the whole SSO (single sign on) process.
Therefore the application has to be started by providing a valid authentication token.
This token is a JWT (JSON Web Token).

.gridFailureInformationData application is called by the *portal* application. The User is already logged in
[plantuml]
....
actor User
participant PortalFrontend
participant PortalBackend
participant GridFailureInformationDataFrontend
entity GridFailureInformationDataStorage
participant GridFailureInformationDataBackend


User->PortalFrontend: Start GridFailureInformationData(JWT)
PortalFrontend->GridFailureInformationDataFrontend: nav. to frontend-URL with JWT
GridFailureInformationDataFrontend->GridFailureInformationDataStorage: Extract JWT and store token in session
... some delay ...
GridFailureInformationDataFrontend->GridFailureInformationDataBackend: Call any secured service with JWT
group Call secured service

    GridFailureInformationDataBackend->PortalBackend: "/checkAut(JWT)"
    group Authorization succeeded
        GridFailureInformationDataBackend->GridFailureInformationDataBackend: run service
        GridFailureInformationDataBackend->GridFailureInformationDataFrontend: return service result
    end
    group Authorization failed
        GridFailureInformationDataBackend->GridFailureInformationDataFrontend: return HTTP Code 401
    end
end
....

=== Publish failure informations

There are different possibilities to publish a failure information.


==== Publish to the SIT-Web-Components for the internet (map and table)
.publish failure information to SIT-Web-Component
[plantuml]
....
participant SIT_BE
participant SIT_WEB_Cache
participant SIT_WEB_FE
participant User

Group scheduled job (i.e. every minute)
    SIT_BE -> SIT_BE: get failure information with status "published"
    SIT_BE -> SIT_WEB_Cache: [POST]<Web-Cache>/internal-sit
    SIT_WEB_Cache -> SIT_WEB_Cache: store in memory

end


Group SIT-WEB-Map-Component / SIT-WEB-Table-Component
    User -> SIT_WEB_FE: Show Failure Infos as map/table
    SIT_WEB_FE -> SIT_WEB_Cache: [GET]<Web-Cache>/public-sit
end
....


As shown in the <<#architecture-overview, architecture overview>> there is a component
to display failure informations as map or as table, available from the internet ("SIT-WEB-FE").
The SIT-Backend has an internal scheduler, which sends all failure informations with the
publishing status "*published*" in a configured time intervall to the SIT-Web-Cache component in the DMZ. SIT-WEB-FE takes
this as data for the display.

==== Publish to different publication channels

.publish failure information to publication channels
[plantuml]
....
actor User
participant SIT_FE
participant SIT_BE
participant RabbitMQ
participant Export_Job

Export_Job -> RabbitMQ: Register as Message Consumer

User -> SIT_FE: Publish qualified failure info
SIT_FE -> SIT_BE: Update?saveForPublish=true
SIT_BE -> SIT_BE: ExportService.exportFailureInfo()

Group iterate over assigned channels
    Group if channel not already published
        Group if channel is mail type
            SIT_BE -> RabbitMQ:: forEach(DistributionGroup)->PrepareMessage->publishMessage()
        end
        Group if change is not mail type
            SIT_BE -> RabbitMQ: publishMessage()
        end
    end
end

Group Listen to mail-queue (MailExporter)
    Export_Job -> RabbitMQ: getMessage()
    Export_Job -> Export_Job: emailService.sendMail()
end

Group List to stoerungsauskunft_export-queue
    Export_Job -> RabbitMQ: getMessage()
    Export_Job -> Export_Job: importExportService.exportStoerungsauskunftOutage()
end
....
The user wants to publish a failure information. With pressing the button "publish"
the "update"-method of the SIT-BE is called with the parameter "saveForPublish=true".
In this case the SIT-BE calls its own subcomponent "ExportService" which iterates
over all assigned export channels of the failure information. Only if a channel has
not been already published yet, the "ExportService" checks if the current channel
is a so-called "mailType" or not. If not the message is directly send to the
corresponding message queue. If the message is a "mailType" then the mail receiver
and the mail text have to be determined/calculated before the mail can be send to the right message queue.
The different export services are listening to "their" message queues and process
incoming messages on their own way asynchronously.


==== Publish to publication channels


=== Interfaces

==== Import of a GridFailureInformation object via MessageBus

First refer to <<#configuration-section-rabbit_mq, RabbitMQ configuration of the backend>>
to setup the RabbitMQ configuration correctly.

A client, that wants to use the message queue to import data, has to use
the correct queue/channel configuration. In addition, the following values must be entered as message headers:

* "*metaId*" Unique id out of the foreign system.
For each metaId from an external system
only one failure information object is ever created. If an existing metaId is sent again,
the existing object is updated in the GridFailureInformation system.

* "*source*" Short description of the external system, the message comes from.
* "*description*" Short description of the message (not mandatory)
* "*payload*" The message payload is the string of the message JSON

Example of a payload:
[source,json]
----
{
    "branch": "S",
    "city": "Chicago",
    "description": "failure in the system",
    "district": "",
    "failureBegin": "2020-11-19T14:13:15.666Z",
    "housenumber": "10b",
    "latitude": 12.345,
    "longitude": 44,
    "planned": true,
    "postcode": "3456",
    "pressureLevel": null,
    "radiusInMeters": 678,
    "stationDescription": null,
    "stationId": null,
    "street": "Downstreet",
    "voltageLevel": "HS"
}
----

Possible values:

* *branch*: S|W|F|G|TK|ST (Strom, Wasser, Fernwärme, Gas, Telekommunikation, Sekundärtechnik)
* *description*: Discription of the failure
* *planned*: "true" or "false" whether the failure is planned or not
* *failureBegin*: Please provide the timestamp in the given form as "Zulu"-Time
* *postcode*: postal code of the address of the failure
* *city*: City of the failure
* *district*: District of the failure
* *street*: Street of the failure
* *housenumber*: Housenumber of the address of the failure
* *stationId*: Station-ID of the failure (for MD powerfailures)
* *stationDescription*: Description of the failure (for MD powerfailures)
* *latitude/longitude*: Coordinates of the failure
* *radiusInMeters*: Radius of the failure in Meters
* *pressureLevel*: ND|MD|HD (Nieder-, Mittel- oder Hochdruck) for gas-failures
* *voltageLevel*: NS|MS|HS (Nieder-, Mittel- oder Hochspannung) for power failures


The header information and the payload are strictly validated. The validation rules
can be found in the files "*ImportDataDto.java*" and "*ForeignFailureDataDto.java*"
(package "*org.eclipse.openk.gridfailureinformation.viewmodel*).

Please refer to the subproject "*test Import Grid Failures*" of the backend repository, for an example for
sending Data over the message queue.

==== Import of addresses from CSV files
Addresses can be imported from CSV files with the addressImport service.
The import of general addresses and addresses for connections of power, gas and water are supported. Also addresses from power stations are imported.
UTM coordinates from the sources are converted into latitude and longitude.
Addresses with the same id are stored in one data row.

Location and filename can be configured in the * .yaml files, which are located in the JAR file of the addressImport service:

[#configuration-section-services]
_Address Import configuration_

* *adressimport.cleanup* If enabled=true the import service deletes all data from the address table before import.
* *adressimport.cron* cron job parameter
* *adressimport.file.addresses* directory and name of the import file for general addresses
* *adressimport.file.power-connections* directory and name of the import file for addresses with power connections
* *adressimport.file.gas-connections* directory and name of the import file for addresses with gas connections
* *adressimport.file.water-connections* directory and name of the import file for addresses with water connections
* *adressimport.file.power-stations* directory and name of the import file for power stations addresses

* *utm.zoneNumber* Number of UTM zone
* *utm.zoneLetter* Letter of UTM zone

===== Source file requirements

The CSV files have to be *semicolon-separated*. An example for a file is shown below.
Examples for all files which can be imported are located in folder 'addressImport/importFiles'.

Example for CSV file with general addresses:
----
sdo_x1;sdo_y1;g3e_fid;plz;ort;ortsteil;strasse;hausnummer
455430,85;5503137,15;74312595;68647;Biblis;Nordheim;Altrheinstr.;29
455448,63;5503151,64;74400207;68647;Biblis;Nordheim;Altrheinstr.;27
455464,51;5503162,81;74347053;68647;Biblis;Nordheim;Altrheinstr.;25
----

The required structure of the import files is shown in the following tables:

====== General addresses

[options="header"]
|=========================================================
|Number of column|Content|Format
|1|utm coordinates easting|Number(9,2)
|2|utm coordinates northing|Number(10,2)
|3|address id|Integer
|4|postcode|String(30)
|5|community|String(255)
|6|district|String(255)
|7|street|String(255)
|8|housenumber|String(30)
|=========================================================

====== Addresses with power connections

[options="header"]
|=========================================================
|Number of column|Content|Format
|1|utm coordinates easting|Number(9,2)
|2|utm coordinates northing|Number(10,2)
|3|address id|Integer
|4|postcode|String(30)
|5|community|String(255)
|6|district|String(255)
|7|street|String(255)
|8|housenumber|String(30)
|9|power station number|String(30)
|=========================================================

====== Addresses with gas connections

[options="header"]
|=========================================================
|Number of column|Content|Format
|1|utm coordinates easting|Number(9,2)
|2|utm coordinates northing|Number(10,2)
|3|address id|Integer
|4|postcode|String(30)
|5|community|String(255)
|6|district|String(255)
|7|street|String(255)
|8|housenumber|String(30)
|9|group|String(255)
|=========================================================

====== Addresses with water connections
[options="header"]
|=========================================================
|Number of column|Content|Format
|1|utm coordinates easting|Number(9,2)
|2|utm coordinates northing|Number(10,2)
|3|address id|Integer
|4|postcode|String(30)
|5|community|String(255)
|6|district|String(255)
|7|street|String(255)
|8|housenumber|String(30)
|9|group|String(255)
|=========================================================


====== Addresses for telecommunication connections
[options="header"]
|=========================================================
|Number of column|Content|Format
|1|utm coordinates easting|Number(9,2)
|2|utm coordinates northing|Number(10,2)
|3|address id|Integer
|4|postcode|String(30)
|5|community|String(255)
|6|district|String(255)
|7|street|String(255)
|8|housenumber|String(30)
|9|stationnumber|String(255)
|=========================================================

====== Addresses for district heating connections
[options="header"]
|=========================================================
|Number of column|Content|Format
|1|utm coordinates easting|Number(9,2)
|2|utm coordinates northing|Number(10,2)
|3|address id|Integer
|4|postcode|String(30)
|5|community|String(255)
|6|district|String(255)
|7|street|String(255)
|8|housenumber|String(30)
|9|stationnumber|String(255)
|=========================================================

====== Addresses for power stations
[options="header"]
|=========================================================
|Number of column|Content|Format
|1|utm coordinates easting|Number(9,2)
|2|utm coordinates northing|Number(10,2)
|3|address id|Integer
|4|power station number|String(30)
|5|power station name|String(255)
|=========================================================

==== Configuration of the system

===== DB based configuration


[#configuration-section-rabbit_mq,Configuration of the backend]
===== Configuration of the grid failure information backend


The backend service is configured in the * .yaml files, which are located in the JAR file.

This yml-file can be divided into different configuration profiles.
When starting the backend-service one has the possibility to specify
the active profile

* *spring.datasource* configuration section for the database connection
* *flyway.enabled* If enabled=true then the database migrations
      will automatically performed when starting the application
      (this parameter should normally be set to "false"
* *server.max-http-header-size* Maximum size for the http-headers
* *jwt.tokenHeader* Name of the http-header which carries the authentication-token.
      (should be "Authorization")
* *jwt.useStaticJwt* If set to "true" then the backend will use *jwt.staticJwt*
      as Authorization-token. (This won't work for calls to other modules
      like the Auth'n'Auth-Modul, because the token will be out of date)
* *authNAuthService.ribbon.listOfServers* Here one can configure the base
     url to the Auth'n'Auth-Service
* *contactService.ribbon.listOfServers* Here one can configure the base
     url to the ContactBaseData-Service


[#configuration-section-services]
_Services configuration_

* *services.authNAuth.name* Name for  Auth'n'Auth-Service-Service (for example "authNAuthService")

* *services.contacts.name* Name for ContactBaseData-Service (for example "contactService")
* *services.contacts.communicationType.mobile* Designation of mobile phone number as communication type in ContactBaseData-Service (for example "Mobil")
* *services.contacts.useModuleNameForFilter*  Boolean Parameter to decide whether to filter contacts in contact service by module name in ContactBaseData-Service ("true" / "false")
* *services.contacts.moduleName*  name of module to filter data from ContactBaseData-Service (for example "GridFailureInformation")

[#configuration-section-rabbit_mq]
_RabbitMQ configuration_

* *rabbitmq.host* RabbitMQ-Server (for example "localhost")
* *rabbitmq.port* Port of the RabbitMQ-Server (for example "5672")
* *rabbitmq.username* Username for the technical RabbitMQ user
* *rabbitmq.password* Password the the technical RabbitMQ user
* *rabbitmq.queuename* Queuename for the import queue (will be created by the backend)
* *rabbitmq.routingkey* Routing key for the import queue
* *rabbitmq.exchangename*: Exchange name for the import queue

* *rabbitmq.importExchange*: Exchange name for the import queue
* *rabbitmq.importQueue* Queuename for the import queue (will be created by the backend)
* *rabbitmq.importkey* Routing key for the import queue
* *rabbitmq.exportExchange*: Exchange name for the export queue
* *rabbitmq.exportQueue*: Queuename for the export queue (will be created by the backend)
* *rabbitmq.exportKey*: Exchange name for the export queue

[#configuration-section-settings]
_Global settings_

* *overviewMapInitialZoom*: Initial zoom-factor for the display of the overview map (default is 10)
* *detailMapInitialZoom*: Inital zoom-factor for the display of map in the detail view (for example 10)
* *overviewMapInitialLatitude*: Initial latitude for the overview map (for example 49.656634)
* *overviewMapInitialLongitude*: Initial longitude for the overview map (for example 8.423207)
* *daysInPastToShowClosedInfos*: Days in the past, before a closed failure information will not be shown any more (for example 365)
* *dataExternInitialVisibility*: *show* or *hide* data in the external map or table unless a postcode has been entered

[#configuration-section-mailtemplates]
_Mail settings (Default templates)_


* *emailSubjectPublishInit*: Template for the subject of the publishing e-mail
* *emailContentPublishInit*: Template for the body of the publishing e-mail
* *emailSubjectUpdateInit*: Template for the subject of the update e-mail
* *emailContentUpdateInit*: Template for the body of the update e-mail
* *emailSubjectCompleteInit*: Template for the subject of the completed e-mail
* *emailContentCompleteInit*: Template for the body of the completed e-mail

[#configuration-section-visibility-of-fields]
_Field visibility settings_

Please configure the visibilty of fields in this section using "show" or "hide"

* *visibilityConfiguration.fieldVisibility*: Here you can set the visibility of fields in the detail mask of the Failure
Information
* *visibilityConfiguration.tableInternColumnVisibility*: Here you can set the visibility of column in the
internal table
* *visibilityConfiguration.tableExternColumnVisibility*: Use "show" or "hide" to toggle the visibility of columns
in the external table
* *visibilityConfiguration.mapExternTooltipVisibility*: Here you can define which column shall be shown or hidden
in the tooltips of the external map


_Credentials(Username and Password)_

All credentials in this yml-files are hidden. Environment variables are used to set
them. Get an environment variable in a yml-file this way `${ENVIRONEMT_VARIABLE}`.
To successfully run the backend service either set the environment variable for the using platform
or replace them in the yml-file.

=== CI- and CD-Components

==== GIT-Repository
Backend:
https://git.eclipse.org/c/openk-usermodules/org.eclipse.openk-usermodules.contactBaseData.backend.git/

Frontend:
https://git.eclipse.org/c/openk-usermodules/org.eclipse.openk-usermodules.contactBaseData.frontend.git/

=== Continuous deployment

The continuous deployment is realized on two platforms:

* the development platform (Dev-Environment)
* the quality platform (Q-Environment)

The automatic deployment on both of the environments is
directly linked to the branches on the GIT-repositories:

. "SNAPSHOT" or "DEVELOP"
. "MASTER" or "TRUNC"

The running development is exclusively made on the Snapshot-Branch. Every time
a developer checks in (pushes) code to the repository, an automatic build
starts on the hudson ci-server. If the Snapshot-build is successful, then the result
of that build is directly deployed on the Dev-environment.

At the end of a scrum sprint or when a big user story is realized, all
the code changes are merged from the *Snapshot*-Branch to the *Trunc*.
This automatically triggers the build and the deployment on the
Q-environment.

==== Deployment of the frontend

TODO:

==== Deployment of the backend
TODO:

==== Deployment of the database
The component "Flyway" is used to make to distribute structural
or content related changes to the database.

The database is built out of the scripts in the directory "db/migrations". Every sql
script contains the complete db script for the contact base data database (in different versions).
The highest version number indicates the currently valid script.

<<<

== Design decisions

All architecture decisions are based on the Architecture Committee Handbook. There are no deviations.

<<<

== Risks and Technical Debts

(Currently there aren't any known issues)

<<<

== Glossary

.Abbreviations and glossary terms
[options="header"]
|========================================================
|Short|Long|German|Description
|AC|Architecture Committee|Architektur-Komittee|Gives framework and constraints according to architecture for oK projects.
|CNCU|Central Network Control Unit||
|DAO|Data Access Objects||
|DTO|Data Transfer Object||
|DSO|Distribution System Operator|Verteilnetz-betreiber (VNB)|Manages the distribution network for energy, gas or water.
|EPL|Eclipse Public License||Underlying license model for Eclipse projects like contact-base-data@openK
|ESB|Enterprise Service Bus||Central instance to exchange data to overcome point-to-point connections.
|oK|openKONSEQUENZ|openKONSEQUENZ|Name of the consortium of DSOs
|QC|Quality Committee|Qualitätskomitee|Gives framework and constraints according to quality for oK projects.
|SCADA|Supervisory Control and Data Acquisition|Netzleitsystem|System, that allows DSOs view/control actual parameters of their power grid.
|========================================================