| //// |
| ******************************************************************************* |
| * Copyright (c) 2019 Contributors to the Eclipse Foundation |
| * |
| * See the NOTICE file(s) distributed with this work for additional |
| * information regarding copyright ownership. |
| * |
| * This program and the accompanying materials are made available under the |
| * terms of the Eclipse Public License v. 2.0 which is available at |
| * http://www.eclipse.org/legal/epl-2.0. |
| * |
| * SPDX-License-Identifier: EPL-2.0 |
| ******************************************************************************* |
| //// |
| |
| = openKonsequenz - Architecture of the module 'Grid Failure Information' |
| Michel Alessandrini <michel.alessandrini@pta.de>; Frank Dietrich <frank.dietrich@pta.de>; Simon Reis <simon.reis@pta.de> |
| :revnumber: 0.2 |
| :revdate: 03.07.2020 |
| :revremark: Inital Draft |
| <<< |
| |
| Based on the ARC42-Template (v7.0): |
| |
| * Version: {revnumber} |
| * Date: {revdate} |
| * Status: {revremark} |
| |
| <<< |
| |
| :icons: |
| :source-highlighter: highlightjs |
| :highlightjs-theme: solarized_dark |
| |
| :lang: en |
| :encoding: utf-8 |
| |
| :imagesdir: ../img |
| :iconsdir: ../img/icons |
| |
| :toc: |
| :toclevels: 4 |
| :toc-title: Table of Contents |
| :toc-placement!: |
| :sectanchors: |
| :numbered: |
| |
| toc::[] |
| |
| <<< |
| |
| == Introduction and Goals |
| |
| === Requirements Overview |
| The module 'Grid Failure Information' (in German: Modul 'Störungsinformationstool') |
| informs employees and customers about failures and planned interruptions of supply. |
| The module realizes the whole process from the creation of a failure information, |
| via the publication until the closure of the failure information. |
| |
| It supports the user by the acquisition of data and leads the user through |
| the workflow for processing a failure information. |
| |
| The module has many interfaces to other modules and external systems. |
| |
| The full requirements of the module 'Grid Failure Information' is described in the document |
| |
| * "Anforderungsspezifikation Modul Störungsinformationstool" version 1.3 / 31-07-2019. |
| |
| |
| === Quality Goals |
| The module 'Grid Failure Information' represents a user module that is based on the architecture platform of openKONSEQUENZ. The main quality |
| goals of the platform are: |
| |
| * *Flexibility* The reference platform shall grant that different systems and modules from different vendors/developers can interact and interoperate, and may be exchanged or recombined. |
| * *Availability* All platform modules that are running on the platform can only be as available as the platform same for user modules that are based on platform modules. |
| * *Maintainability* (and testability as part of maintainability) The platform and its platform modules shall be used longer than 15 years. |
| * *Integration performance* New implemented functionality of oK own modules and external modules shall be included fast / automatically. |
| * *Security* The platform and its modules need to underly security-by-design |
| |
| The main quality goals of the core module Contact Base Data are: |
| |
| * *Functionality* The core module must fulfil the functional requirements mentioned in the section before |
| * *Ergonomics* The web interface must be realized according to oK-GUI-Styleguide. |
| * *Good documentation* (i.e. code and architecture documentation) makes code changes easier and automatic |
| tests facilitate rigorous verification. |
| * *Modifiability* (and testability as part of modifiability) |
| * *Integration performance* The core module must be easy integratable in different production environments. |
| |
| The following documents contain the quality goals in detail: |
| |
| * Architecture Committee Handbook v1.6.0 from 10-07-2019 |
| * Quality Committee Handbook v2.0.1 from 15-10-2018 |
| |
| The architecture is based on the AC-Handbook. The quality demands are described in the QC-Handbook. |
| Both specifications were fully complied with in the project, so that a high quality is given. |
| |
| The code quality regarding static code analysis and unit test code coverage on the backend and fronend sides |
| are ensured by the use of sonarqube. The rule set and the qualtity gate are defined by the default, the |
| so called "sonar way". |
| |
| The module 'Grid Failure Information' is part of the Eclipse project 'Eclipse openK User Modules'. |
| This project bases on the Eclipse Public Licence 2.0. |
| |
| === Stakeholders |
| |
| .Stakeholders |
| [options="header"] |
| |========================================================= |
| |Role/Name|Contact|Expectations |
| |Product Owner (represents the Distribution System Operators)|Maike Salbeck, Carsten Otten, Benedikt Herget, Rainer Fuhrmann|The software must fulfil their functional and nonfunctional requirements. |
| |Module Developer|Michel Alessandrini, Jonas Tewolde, Frank Dietrich|All relevant business and technical information must be available for implementing the software. |
| |External Reviewer (represents the AC/QC)|n.n.|The software and the documentation is realized according to the Quality and Architecture Handbook of openKONSEQUENZ. |
| |External Reviewer (represents the Eclipse-Requirements)|n.n.|The software is licensed under the EPL 2.0. It must be validated that all requirements are fulfilled. |
| |System Integrator|n.n.|A documentation for the integration of the module in the DSO specific environments must be available. |
| |========================================================= |
| |
| <<< |
| |
| == Architecture Constraints |
| |
| The main architecture constraints are: |
| |
| * *Public License* The module must be available under the “Eclipse Public License 2.0”. |
| * *Standardization* The module must use the reference platform. |
| * *Availability* The source code of the module must be accessible to any interested person/company. |
| |
| Therefore the project is published under the following repositories: |
| |
| * https://git.eclipse.org/r/openk-usermodules/org.eclipse.openk-usermodules.gridFailureInformation.backend |
| * https://git.eclipse.org/r/openk-usermodules/org.eclipse.openk-usermodules.gridFailureInformation.frontend |
| * https://git.eclipse.org/r/openk-usermodules/org.eclipse.openk-usermodules.gridFailureInformation.documentation |
| |
| |
| === Technical Constraints |
| |
| The following technical constraints are given: |
| |
| .Technical Contraints |
| [options="header"] |
| |======================================================== |
| |Component|Constraints |
| |Base components of the reference platform |
| a|* Application Server Tomcat |
| * JPA EclipseLink |
| * Database PostgreSQL |
| |
| |Programming language frontend |
| a|* Angular |
| * Bootstrap |
| * jQuery |
| * REST/JSON Interfaces |
| |
| |GUI design |
| a|* According to oK-GUI-Styleguide |
| |
| |Java QA environment |
| a| * Sonarqube 5.6.6 |
| |
| |Programming language |
| a|* Backend: Java 1.8 |
| * Frontend: Angular 7+ (Javascript, Typescript, HTML5, CSS3) |
| |
| |IDE |
| a|* Not restricted (Eclipse, IntelliJ, Microsoft Developer Studio, Microsoft Visual Code ...) |
| |
| |Build system |
| a|* Backend: Maven |
| * Frontend: NodeJS + Angular/cli |
| |
| |Libraries, frameworks, components |
| a|* Used Libraries/Frameworks have to be compatible to the Eclipse Public License |
| |
| |Architecture Documentation |
| a|* According ARC42-Template |
| |======================================================== |
| |
| |
| === Technical Dependencies |
| |
| ==== Modules |
| The following modules are required to use the 'Grid Failure Information': |
| |
| .Modules |
| [options="header"] |
| |========================================================= |
| |Name of the module|Purpose|Status of the module|status |
| |'Auth&Auth'|Authentification and Authorization|available|required |
| |'Contact base data'|Contact Base Data|available|required |
| |'eLogbook'|Log Book|available|optional |
| |'Planned Grid Measures'|Planned Grid Measures|available|optional |
| |'CIM-Cache'|Interface to the SCADA system|available|optional |
| |========================================================= |
| |
| |
| ==== Libraries |
| |
| The following libraries are used: |
| TODO: Am Ende die Bibliotheken aktualisieren |
| |
| .Libraries |
| [options="header"] |
| |========================================================= |
| |Name of the library|Version|Artefact-id|Usage|License|Tier |
| |
| |
| |org.springframework.boot.spring-boot-starter-parent|2.2.1.RELEASE |
| a| |
| ||Apache License 2.0|Backend |
| |
| |org.springframework.boot.spring-boot-starter-data-jpa|2.2.1.RELEASE |
| a| |
| ||Apache License 2.0|Backend |
| |
| |org.springframework.boot.spring-boot-starter-oauth2-client|2.2.1.RELEASE |
| a| |
| ||Apache License 2.0|Backend |
| |
| |org.springframework.boot.spring-boot-starter-security|2.2.1.RELEASE |
| a| |
| ||Apache License 2.0|Backend |
| |
| |org.springframework.boot.spring-boot-starter-web|2.2.1.RELEASE |
| a| |
| ||Apache License 2.0|Backend |
| |
| |org.flywaydb.flyway-core|6.0.8 |
| a| |
| ||Apache License 2.0|Backend |
| |
| |org.springframework.cloud.spring-cloud-starter-openfeign|2.2.0.RELEASE |
| a| |
| ||Apache License 2.0|Backend |
| |
| |org.springframework.cloud.spring-cloud-starter-netflix-ribbon|2.2.0.RELEASE |
| a| |
| ||Apache License 2.0|Backend |
| |
| |org.keycloak.keycloak-core|3.4.2_Final |
| a| |
| ||Apache License 2.0|Backend |
| |
| |org.postgresql.postgresql|42.2.8 |
| a| |
| ||New BSD License|Backend |
| |
| |org.projectlombok.lombok|1.18.10 |
| a| |
| ||MIT|Backend |
| |
| |org.mapstruct.mapstruct-processor|1.2.0.Final |
| a| |
| ||Apache License 2.0|Backend |
| |
| |io.jsonwebtoken.jjwt|0.9.1 |
| a| |
| ||Apache License 2.0|Backend |
| |
| |io.springfox.springfox-swagger2|2.9.2 |
| a| |
| ||Apache License 2.0|Backend |
| |
| |io.springfox.springfox-swagger-ui|2.9.2 |
| a| |
| ||Apache License 2.0|Backend |
| |
| |org.springframework.boot.spring-boot-starter-test|2.2.1.RELEASE |
| a| |
| ||Apache License 2.0|Backend |
| |
| |org.springframework.security.spring-security-test|5.2.1.RELEASE |
| a| |
| ||Apache License 2.0|Backend |
| |
| |org.powermock.powermock-reflect|2.0.0 |
| a| |
| ||Apache License 2.0|Backend |
| |
| |com.h2database.h2|1.4.200 |
| a| |
| ||EPL|Backend |
| |
| |org.springframework.cloud.spring-cloud-dependencies|Hoxton.RELEASE |
| a| |
| ||Apache License 2.0|Backend |
| |
| |org.springframework.boot.spring-boot-maven-plugin|2.2.1.RELEASE |
| a| |
| ||Apache License 2.0|Backend |
| |
| |org.jacoco.jacoco-maven-plugin|0.7.9 |
| a| |
| ||EPL 2.0|Backend |
| |
| |org.sonarsource.scanner.maven.sonar-maven-plugin|3.2 |
| a| |
| ||LGPL 3.0|Backend |
| |
| |org.asciidoctor.asciidoctor-maven-plugin|1.5.3 |
| a| |
| ||Apache License 2.0|Backend |
| |
| |
| |org.jruby.jruby-complete|9.0.0.0 |
| a| |
| ||EPL 2.0|Backend |
| |
| |org.asciidoctor.asciidoctorj-pdf|1.5.0-alpha.11 |
| a| |
| ||Apache 2.0|Backend |
| |
| |
| |org.asciidoctor.asciidoctorj|1.5.4 |
| a| |
| ||Apache 2.0|Backend |
| |
| |
| |org.asciidoctor.asciidoctorj-pdf|1.5.0-alpha.11 |
| a| |
| ||Apache 2.0|Backend |
| |
| |
| |org.asciidoctor.asciidoctorj-diagram|1.5.4.1 |
| a| |
| ||Apache 2.0|Backend |
| |
| |
| |Angular Font Awesome|3.1.2 |
| a| |
| ||MIT License|Frontend |
| |
| |
| |@auth0/angular-jwt|3.0.1 |
| a| |
| ||MIT License|Frontend |
| |
| |
| |font-awesome|4.7.0 |
| a| |
| ||MIT License|Frontend |
| |
| |
| |@ngrx/core|1.2.0 |
| a| |
| ||MIT License|Frontend |
| |
| |
| |@ngrx/effects|8-2-0 |
| a| |
| ||MIT License|Frontend |
| |
| |
| |@ngrx/store|8.3.0 |
| a| |
| ||MIT License|Frontend |
| |
| |
| |@ngrx/store-devtools|8.2.0 |
| a| |
| ||MIT License|Frontend |
| |
| |
| |@ngx-translate/core|11.0.1 |
| a| |
| ||MIT License|Frontend |
| |
| |
| |@ngx-translate/http-loader|4.0.0 |
| a| |
| ||MIT License|Frontend |
| |
| |
| |ag-grid-angular|21.2.1 |
| a| |
| ||MIT License|Frontend |
| |
| |
| |ag-grid-community|21.2.1 |
| a| |
| ||MIT License|Frontend |
| |
| |
| |angular2-notifications|2.0.0 |
| a| |
| ||MIT License|Frontend |
| |
| |
| |bootstrap|4.4.1 |
| a| |
| ||MIT License|Frontend |
| |
| |
| |jquery|3.4.1 |
| a| |
| ||MIT License|Frontend |
| |
| |
| |classlist.js|1.1.20150312 |
| a| |
| ||MIT License|Frontend |
| |
| |
| |core-js|3.2.1 |
| a| |
| ||MIT License|Frontend |
| |
| |
| |moment|2.24.0 |
| a| |
| ||MIT License|Frontend |
| |
| |
| |ng2-popover|0.0.14 |
| a| |
| ||MIT License|Frontend |
| |
| |
| |ngrx-forms|5.2.1 |
| a| |
| ||MIT License|Frontend |
| |
| |
| |npm-install-peers|1.2.1 |
| a| |
| ||MIT License|Frontend |
| |
| |
| |reselect|4.0.0 |
| a| |
| ||MIT License|Frontend |
| |
| |
| |rxjs|6.5.3 |
| a| |
| ||MIT License|Frontend |
| |
| |
| |rxjs-compat|6.5.4 |
| a| |
| ||MIT License|Frontend |
| |
| |
| |ts-helpers|1.1.2 |
| a| |
| ||MIT License|Frontend |
| |
| |
| |tslib|1.10.0 |
| a| |
| ||MIT License|Frontend |
| |
| |
| |web-animations-js|2.3.2 |
| a| |
| ||MIT License|Frontend |
| |
| |
| |zone.js|0.10.1 |
| a| |
| ||MIT License|Frontend |
| |
| |
| |@swimlane/ngx-datatable|15.0.2 |
| a| |
| ||MIT License|Frontend |
| |
| |
| |puppeteer|2.0.0 |
| a| |
| ||MIT License|Frontend |
| |
| |
| |ngx-toastr|11.2.1 |
| a| |
| ||MIT License|Frontend |
| |
| |popper.js|1.16.0 |
| a| |
| ||MIT License|Frontend |
| |
| |@ng-bootstrap|5.1.5 |
| a| |
| ||MIT License|Frontend |
| |
| |
| |========================================================= |
| |
| <<< |
| |
| == System Scope and Context |
| |
| === Business Context |
| |
| The core module 'Grid Failure Information' communicates via Restful Webservices with the follwowing modules: |
| |
| * *Core Module 'Auth & Auth'* The 'Grid Failure Information' can only be used by authorized users. |
| Therefore, it is essential to invoke the module 'Auth & Auth' for authorization and authentication |
| purposes. |
| |
| * *Core Module 'Contact base data'* The user management can only by done by the module contact base data. |
| Therefore, it is essential to user the module ''Contact base data' for managing all users and customers. |
| |
| * *Core Module 'CIM-Cache'* The 'Grid Failure Information' can only communicate via the CIM-Cachewith the SCADA system . |
| Therefore, it is essential to invoke the module 'CIM-Cache' for getting information from the SCADA system. |
| |
| * *User Module 'eLogbook'* The 'Grid Failure Information' can import failure information from the module 'eLogbook'. |
| |
| * *User Module 'Planned Grid Measures'* The 'Grid Failure Information' can import failure information from the module 'Planned Grid Measures'. |
| |
| |
| === Technical Context |
| |
| The following aspects have to be taken into account for external communication of the module 'Grid Failure Information': |
| |
| * RESTful web services are used as interface-technology. |
| * Each external interface (interfaces between modules or external systems) has to be documented. |
| * Dependencies of modules to services realized by other modules have to be specified and documented explicitly. |
| |
| The interfaces of the module 'Grid Failure Information' are described in the interface documentation. |
| TODO: Link zur interface documentation einfügen |
| |
| === Solution Strategy |
| |
| The module 'Grid Failure Information' bases on a small microservice architecture, including |
| an asynchronous messaging system. |
| |
| <<< |
| |
| == Building Block View |
| |
| === Whitebox Overall System |
| |
| The module 'grid failure information' contains several components: |
| |
| . *SIT-Web-FE* - This component (SPA with Angular) provides two HTML pages: a table and a map |
| with failure information. This component is in the DMZ and can be called up from the Internet. |
| . *SIT-Web-Cache-Service* - The SIT-Web-Cache-Service is a service which receives the published |
| grid failures in the form of a JSON from the SIT-BE and makes them available to the SIT-Web-FE |
| via a ReST interface. This component is implemented with Angular / Typescript. Because this |
| component is only equipped with freely accessible data, no security is necessary here. The |
| SIT-Web-Cache-Service, like the SIT-Web-FE, is situated in the DMZ. |
| . *SIT-FE* - |
| This component (SPA with Angular) provides the user interface for the failure information application. The SIT-FE |
| receives its authentication when it is called from the PortalFE in the form of a JWT. |
| . *SIT_BE* - The SIT-BE (Java Spring Boot Microservice) provides all CRUD services in the form of |
| ReST services that the frontend requires. The SIT-BE is the only component that has |
| access to the database. Every call is authorized against the PortalBE. This microservice also includes |
| the JobManager subcomponent. |
| . *JobManager* - The various imports and exports are controlled via the JobManager. The JobManager |
| knows all available import and export jobs and can control them via the internal message bus |
| (RabbitMQ internal). Because the communication between the JobManager and the jobs takes place |
| via the message bus, the JM does not need to know their configuration and URLs. For this, the JM must |
| ensure synchronization (incl. timeout behavior) for asynchronous message communication. |
| . *Import- and export jobs* - All jobs provide a uniform (MessageBus) interface to start a job, |
| return the result of a job, or to provide information about the respective job (name, version, timeout, status, etc.). |
| The job manager cyclically requests all configured jobs to send current status information to it. |
| The interfaces of the individual jobs "outside" can be very different (file system, Internet, message queue, etc.) |
| |
| [#architecture-overview] |
| .Architecture of the grid failure information tool |
| [options="header"] |
| image::architectureSIT.png[] |
| |
| |
| |
| === Level 2 |
| |
| ==== GridFailureInformationFE (frontend tier) |
| |
| The frontend component implements the concept of a single-page application (SPA). The framework used is Angular5. |
| |
| It divides the gridFailureInformationFE into three layers: |
| |
| . *Components* - The components (pages, lists, dialogs, common comp.) represent the presentation layer and the control layer. A component contains the control logic (.ts-file), an HTML-fragment as presentation description (.html-file) and a style definition (.css-file). |
| . *Services* - The service component communicates with the interfaces of the backend via HTTP requests by using the model component. |
| . *Model* - The model corresponds to the view-model of the backend tier. |
| |
| .Frontend tier |
| [options="header"] |
| [plantuml] |
| ---- |
| |
| node gridFailureInformation_Frontend { |
| component Model |
| |
| |
| node Components { |
| component "Pages" |
| component Lists |
| component "Common Components" |
| } |
| |
| component Services |
| |
| Components --> Services |
| Components --> Model |
| Services --> Model |
| } |
| |
| node "Grid Failure Information Backend (simplified)" { |
| component RestService |
| component ViewModel_API__DTO |
| } |
| |
| Services .. RestService |
| Model .. ViewModel_API__DTO |
| RestService --> ViewModel_API__DTO |
| ---- |
| |
| |
| ==== grid-failure-information (backend tier) |
| |
| The backend tier contains five components which can be summarized in three layers: |
| |
| . *Presentation layer* - Represented by |
| .. REST-Srv |
| .. View model/DTO |
| . *Controller layer* - Represented by |
| .. Controller |
| .. Service |
| . *Model layer* - Represented by |
| .. Repository |
| .. Model |
| |
| |
| |
| .Backend tier |
| [options="header"] |
| [plantuml] |
| ---- |
| |
| node "Grid Failure Information Backend" { |
| component Model |
| |
| |
| component RestService |
| |
| component ViewModel_DTO |
| |
| component Controller |
| |
| component Service |
| |
| component Repository |
| |
| RestService --> ViewModel_DTO |
| RestService --> Controller |
| Controller --> Service |
| Service --> Repository |
| Repository --> Model |
| } |
| |
| node DBMS { |
| component GridFailureInformationDB |
| } |
| |
| Repository --> ContactBaseDataDB |
| ---- |
| |
| ==== ContactBaseData-DB (database tier) |
| |
| The ContactBaseData-DB is realized as a relational database system. |
| |
| .Database tier |
| [options="header"] |
| [plantuml] |
| ---- |
| |
| node DBMS { |
| component ContactBaseDataDB |
| } |
| ---- |
| |
| |
| ==== Program Configuration |
| |
| <<< |
| |
| == Runtime view |
| |
| === Login / authentication |
| |
| |
| There is no login page, since the openK-Portal-Application is responsible for authentication and |
| the whole SSO (single sign on) process. |
| Therefore the application has to be started by providing a valid authentication token. |
| This token is a JWT (JSON Web Token). |
| |
| .gridFailureInformationData application is called by the *portal* application. The User is already logged in |
| [plantuml] |
| .... |
| actor User |
| participant PortalFrontend |
| participant PortalBackend |
| participant GridFailureInformationDataFrontend |
| entity GridFailureInformationDataStorage |
| participant GridFailureInformationDataBackend |
| |
| |
| User->PortalFrontend: Start GridFailureInformationData(JWT) |
| PortalFrontend->GridFailureInformationDataFrontend: nav. to frontend-URL with JWT |
| GridFailureInformationDataFrontend->GridFailureInformationDataStorage: Extract JWT and store token in session |
| ... some delay ... |
| GridFailureInformationDataFrontend->GridFailureInformationDataBackend: Call any secured service with JWT |
| group Call secured service |
| |
| GridFailureInformationDataBackend->PortalBackend: "/checkAut(JWT)" |
| group Authorization succeeded |
| GridFailureInformationDataBackend->GridFailureInformationDataBackend: run service |
| GridFailureInformationDataBackend->GridFailureInformationDataFrontend: return service result |
| end |
| group Authorization failed |
| GridFailureInformationDataBackend->GridFailureInformationDataFrontend: return HTTP Code 401 |
| end |
| end |
| .... |
| |
| === Publish failure informations |
| |
| There are different possibilities to publish a failure information. |
| |
| |
| ==== Publish to the SIT-Web-Components for the internet (map and table) |
| .publish failure information to SIT-Web-Component |
| [plantuml] |
| .... |
| participant SIT_BE |
| participant SIT_WEB_Cache |
| participant SIT_WEB_FE |
| participant User |
| |
| Group scheduled job (i.e. every minute) |
| SIT_BE -> SIT_BE: get failure information with status "published" |
| SIT_BE -> SIT_WEB_Cache: [POST]<Web-Cache>/internal-sit |
| SIT_WEB_Cache -> SIT_WEB_Cache: store in memory |
| |
| end |
| |
| |
| Group SIT-WEB-Map-Component / SIT-WEB-Table-Component |
| User -> SIT_WEB_FE: Show Failure Infos as map/table |
| SIT_WEB_FE -> SIT_WEB_Cache: [GET]<Web-Cache>/public-sit |
| end |
| .... |
| |
| |
| As shown in the <<#architecture-overview, architecture overview>> there is a component |
| to display failure informations as map or as table, available from the internet ("SIT-WEB-FE"). |
| The SIT-Backend has an internal scheduler, which sends all failure informations with the |
| publishing status "*published*" in a configured time intervall to the SIT-Web-Cache component in the DMZ. SIT-WEB-FE takes |
| this as data for the display. |
| |
| ==== Publish to different publication channels |
| |
| .publish failure information to publication channels |
| [plantuml] |
| .... |
| actor User |
| participant SIT_FE |
| participant SIT_BE |
| participant RabbitMQ |
| participant Export_Job |
| |
| Export_Job -> RabbitMQ: Register as Message Consumer |
| |
| User -> SIT_FE: Publish qualified failure info |
| SIT_FE -> SIT_BE: Update?saveForPublish=true |
| SIT_BE -> SIT_BE: ExportService.exportFailureInfo() |
| |
| Group iterate over assigned channels |
| Group if channel not already published |
| Group if channel is mail type |
| SIT_BE -> RabbitMQ:: forEach(DistributionGroup)->PrepareMessage->publishMessage() |
| end |
| Group if change is not mail type |
| SIT_BE -> RabbitMQ: publishMessage() |
| end |
| end |
| end |
| |
| Group Listen to mail-queue (MailExporter) |
| Export_Job -> RabbitMQ: getMessage() |
| Export_Job -> Export_Job: emailService.sendMail() |
| end |
| |
| Group List to stoerungsauskunft_export-queue |
| Export_Job -> RabbitMQ: getMessage() |
| Export_Job -> Export_Job: importExportService.exportStoerungsauskunftOutage() |
| end |
| .... |
| The user wants to publish a failure information. With pressing the button "publish" |
| the "update"-method of the SIT-BE is called with the parameter "saveForPublish=true". |
| In this case the SIT-BE calls its own subcomponent "ExportService" which iterates |
| over all assigned export channels of the failure information. Only if a channel has |
| not been already published yet, the "ExportService" checks if the current channel |
| is a so-called "mailType" or not. If not the message is directly send to the |
| corresponding message queue. If the message is a "mailType" then the mail receiver |
| and the mail text have to be determined/calculated before the mail can be send to the right message queue. |
| The different export services are listening to "their" message queues and process |
| incoming messages on their own way asynchronously. |
| |
| |
| |
| ==== Publish to publication channels |
| |
| |
| === Interfaces |
| |
| ==== Import of a GridFailureInformation object via MessageBus |
| |
| First refer to <<#configuration-section-rabbit_mq, RabbitMQ configuration of the backend>> |
| to setup the RabbitMQ configuration correctly. |
| |
| A client, that wants to use the message queue to import data, has to use |
| the correct queue/channel configuration. In addition, the following values must be entered as message headers: |
| |
| * "*metaId*" Unique id out of the foreign system. |
| For each metaId from an external system |
| only one failure information object is ever created. If an existing metaId is sent again, |
| the existing object is updated in the GridFailureInformation system. |
| |
| * "*source*" Short description of the external system, the message comes from. |
| * "*description*" Short description of the message (not mandatory) |
| * "*payload*" The message payload is the string of the message JSON |
| |
| Example of a payload: |
| [source,json] |
| ---- |
| { |
| "branch": "S", |
| "city": "Chicago", |
| "description": "failure in the system", |
| "district": "", |
| "failureBegin": "2020-11-19T14:13:15.666Z", |
| "housenumber": "10b", |
| "latitude": 12.345, |
| "longitude": 44, |
| "planned": true, |
| "postcode": "3456", |
| "pressureLevel": null, |
| "radiusInMeters": 678, |
| "stationDescription": null, |
| "stationId": null, |
| "street": "Downstreet", |
| "voltageLevel": "HS" |
| } |
| ---- |
| |
| Possible values: |
| |
| * *branch*: S|W|F|G|TK|ST (Strom, Wasser, Fernwärme, Gas, Telekommunikation, Sekundärtechnik) |
| * *description*: Discription of the failure |
| * *planned*: "true" or "false" whether the failure is planned or not |
| * *failureBegin*: Please provide the timestamp in the given form as "Zulu"-Time |
| * *postcode*: postal code of the address of the failure |
| * *city*: City of the failure |
| * *district*: District of the failure |
| * *street*: Street of the failure |
| * *housenumber*: Housenumber of the address of the failure |
| * *stationId*: Station-ID of the failure (for MD powerfailures) |
| * *stationDescription*: Description of the failure (for MD powerfailures) |
| * *latitude/longitude*: Coordinates of the failure |
| * *radiusInMeters*: Radius of the failure in Meters |
| * *pressureLevel*: ND|MD|HD (Nieder-, Mittel- oder Hochdruck) for gas-failures |
| * *voltageLevel*: NS|MS|HS (Nieder-, Mittel- oder Hochspannung) for power failures |
| |
| |
| The header information and the payload are strictly validated. The validation rules |
| can be found in the files "*ImportDataDto.java*" and "*ForeignFailureDataDto.java*" |
| (package "*org.eclipse.openk.gridfailureinformation.viewmodel*). |
| |
| Please refer to the subproject "*test Import Grid Failures*" of the backend repository, for an example for |
| sending Data over the message queue. |
| |
| ==== Import of addresses from CSV files |
| Addresses can be imported from CSV files with the addressImport service. |
| The import of general addresses and addresses for connections of power, gas and water are supported. Also addresses from power stations are imported. |
| UTM coordinates from the sources are converted into latitude and longitude. |
| Addresses with the same id are stored in one data row. |
| |
| Location and filename can be configured in the * .yaml files, which are located in the JAR file of the addressImport service: |
| |
| [#configuration-section-services] |
| _Address Import configuration_ |
| |
| * *adressimport.cleanup* If enabled=true the import service deletes all data from the address table before import. |
| * *adressimport.cron* cron job parameter |
| * *adressimport.file.addresses* directory and name of the import file for general addresses |
| * *adressimport.file.power-connections* directory and name of the import file for addresses with power connections |
| * *adressimport.file.gas-connections* directory and name of the import file for addresses with gas connections |
| * *adressimport.file.water-connections* directory and name of the import file for addresses with water connections |
| * *adressimport.file.power-stations* directory and name of the import file for power stations addresses |
| |
| * *utm.zoneNumber* Number of UTM zone |
| * *utm.zoneLetter* Letter of UTM zone |
| |
| ===== Source file requirements |
| |
| The CSV files have to be *semicolon-separated*. An example for a file is shown below. |
| Examples for all files which can be imported are located in folder 'addressImport/importFiles'. |
| |
| Example for CSV file with general addresses: |
| ---- |
| sdo_x1;sdo_y1;g3e_fid;plz;ort;ortsteil;strasse;hausnummer |
| 455430,85;5503137,15;74312595;68647;Biblis;Nordheim;Altrheinstr.;29 |
| 455448,63;5503151,64;74400207;68647;Biblis;Nordheim;Altrheinstr.;27 |
| 455464,51;5503162,81;74347053;68647;Biblis;Nordheim;Altrheinstr.;25 |
| ---- |
| |
| The required structure of the import files is shown in the following tables: |
| |
| ====== General addresses |
| |
| [options="header"] |
| |========================================================= |
| |Number of column|Content|Format |
| |1|utm coordinates easting|Number(9,2) |
| |2|utm coordinates northing|Number(10,2) |
| |3|address id|Integer |
| |4|postcode|String(30) |
| |5|community|String(255) |
| |6|district|String(255) |
| |7|street|String(255) |
| |8|housenumber|String(30) |
| |========================================================= |
| |
| ====== Addresses with power connections |
| |
| [options="header"] |
| |========================================================= |
| |Number of column|Content|Format |
| |1|utm coordinates easting|Number(9,2) |
| |2|utm coordinates northing|Number(10,2) |
| |3|address id|Integer |
| |4|postcode|String(30) |
| |5|community|String(255) |
| |6|district|String(255) |
| |7|street|String(255) |
| |8|housenumber|String(30) |
| |9|power station number|String(30) |
| |========================================================= |
| |
| ====== Addresses with gas connections |
| |
| [options="header"] |
| |========================================================= |
| |Number of column|Content|Format |
| |1|utm coordinates easting|Number(9,2) |
| |2|utm coordinates northing|Number(10,2) |
| |3|address id|Integer |
| |4|postcode|String(30) |
| |5|community|String(255) |
| |6|district|String(255) |
| |7|street|String(255) |
| |8|housenumber|String(30) |
| |9|group|String(255) |
| |========================================================= |
| |
| ====== Addresses with water connections |
| [options="header"] |
| |========================================================= |
| |Number of column|Content|Format |
| |1|utm coordinates easting|Number(9,2) |
| |2|utm coordinates northing|Number(10,2) |
| |3|address id|Integer |
| |4|postcode|String(30) |
| |5|community|String(255) |
| |6|district|String(255) |
| |7|street|String(255) |
| |8|housenumber|String(30) |
| |9|group|String(255) |
| |========================================================= |
| |
| |
| ====== Addresses for telecommunication connections |
| [options="header"] |
| |========================================================= |
| |Number of column|Content|Format |
| |1|utm coordinates easting|Number(9,2) |
| |2|utm coordinates northing|Number(10,2) |
| |3|address id|Integer |
| |4|postcode|String(30) |
| |5|community|String(255) |
| |6|district|String(255) |
| |7|street|String(255) |
| |8|housenumber|String(30) |
| |9|stationnumber|String(255) |
| |========================================================= |
| |
| ====== Addresses for district heationg connections |
| [options="header"] |
| |========================================================= |
| |Number of column|Content|Format |
| |1|utm coordinates easting|Number(9,2) |
| |2|utm coordinates northing|Number(10,2) |
| |3|address id|Integer |
| |4|postcode|String(30) |
| |5|community|String(255) |
| |6|district|String(255) |
| |7|street|String(255) |
| |8|housenumber|String(30) |
| |9|stationnumber|String(255) |
| |========================================================= |
| |
| ====== Addresses for power stations |
| [options="header"] |
| |========================================================= |
| |Number of column|Content|Format |
| |1|utm coordinates easting|Number(9,2) |
| |2|utm coordinates northing|Number(10,2) |
| |3|address id|Integer |
| |4|power station number|String(30) |
| |5|power station name|String(255) |
| |========================================================= |
| |
| === Deployment of the application components |
| |
| ==== Deployment of the frontend |
| |
| TODO: |
| |
| ==== Deployment of the backend |
| TODO: |
| |
| ==== Deployment of the database |
| The component "Flyway" is used to make to distribute structural |
| or content related changes to the database. |
| |
| The database is built out of the scripts in the directory "db/migrations". Every sql |
| script contains the complete db script for the contact base data database (in different versions). |
| The highest version number indicates the currently valid script. |
| |
| ==== Configuration of the system |
| |
| ===== DB based configuration |
| |
| |
| [#configuration-section-rabbit_mq,Configuration of the backend] |
| ===== Configuration of the grid failure information backend |
| |
| |
| The backend service is configured in the * .yaml files, which are located in the JAR file. |
| |
| This yml-file can be divided into different configuration profiles. |
| When starting the backend-service one has the possibility to specify |
| the active profile |
| |
| * *spring.datasource* configuration section for the database connection |
| * *flyway.enabled* If enabled=true then the database migrations |
| will automatically performed when starting the application |
| (this parameter should normally be set to "false" |
| * *server.max-http-header-size* Maximum size for the http-headers |
| * *jwt.tokenHeader* Name of the http-header which carries the authentication-token. |
| (should be "Authorization") |
| * *jwt.useStaticJwt* If set to "true" then the backend will use *jwt.staticJwt* |
| as Authorization-token. (This won't work for calls to other modules |
| like the Auth'n'Auth-Modul, because the token will be out of date) |
| * *authNAuthService.ribbon.listOfServers* Here one can configure the base |
| url to the Auth'n'Auth-Service |
| * *contactService.ribbon.listOfServers* Here one can configure the base |
| url to the ContactBaseData-Service |
| |
| |
| [#configuration-section-services] |
| _Services configuration_ |
| |
| * *services.authNAuth.name* Name for Auth'n'Auth-Service-Service (for example "authNAuthService") |
| |
| * *services.contacts.name* Name for ContactBaseData-Service (for example "contactService") |
| * *services.contacts.communicationType.mobile* Designation of mobile phone number as communication type in ContactBaseData-Service (for example "Mobil") |
| * *services.contacts.useModuleNameForFilter* Boolean Parameter to decide whether to filter contacts in contact service by module name in ContactBaseData-Service ("true" / "false") |
| * *services.contacts.moduleName* name of module to filter data from ContactBaseData-Service (for example "GridFailureInformation") |
| |
| [#configuration-section-rabbit_mq] |
| _RabbitMQ configuration_ |
| |
| * *rabbitmq.host* RabbitMQ-Server (for example "localhost") |
| * *rabbitmq.port* Port of the RabbitMQ-Server (for example "5672") |
| * *rabbitmq.username* Username for the technical RabbitMQ user |
| * *rabbitmq.password* Password the the technical RabbitMQ user |
| * *rabbitmq.queuename* Queuename for the import queue (will be created by the backend) |
| * *rabbitmq.routingkey* Routing key for the import queue |
| * *rabbitmq.exchangename*: Exchange name for the import queue |
| |
| * *rabbitmq.importExchange*: Exchange name for the import queue |
| * *rabbitmq.importQueue* Queuename for the import queue (will be created by the backend) |
| * *rabbitmq.importkey* Routing key for the import queue |
| * *rabbitmq.exportExchange*: Exchange name for the export queue |
| * *rabbitmq.exportQueue*: Queuename for the export queue (will be created by the backend) |
| * *rabbitmq.exportKey*: Exchange name for the export queue |
| |
| _Credentials(Username and Password)_ |
| |
| All credentials in this yml-files are hidden. Environment variables are used to set |
| them. Get an environment variable in a yml-file this way `${ENVIRONEMT_VARIABLE}`. |
| To successfully run the backend service either set the environment variable for the using platform |
| or replace them in the yml-file. |
| |
| === CI- and CD-Components |
| |
| ==== GIT-Repository |
| Backend: |
| https://git.eclipse.org/c/openk-usermodules/org.eclipse.openk-usermodules.contactBaseData.backend.git/ |
| |
| Frontend: |
| https://git.eclipse.org/c/openk-usermodules/org.eclipse.openk-usermodules.contactBaseData.frontend.git/ |
| |
| === Continuous deployment |
| |
| The continuous deployment is realized on two platforms: |
| |
| * the development platform (Dev-Environment) |
| * the quality platform (Q-Environment) |
| |
| The automatic deployment on both of the environments is |
| directly linked to the branches on the GIT-repositories: |
| |
| . "SNAPSHOT" or "DEVELOP" |
| . "MASTER" or "TRUNC" |
| |
| The running development is exclusively made on the Snapshot-Branch. Every time |
| a developer checks in (pushes) code to the repository, an automatic build |
| starts on the hudson ci-server. If the Snapshot-build is successful, then the result |
| of that build is directly deployed on the Dev-environment. |
| |
| At the end of a scrum sprint or when a big user story is realized, all |
| the code changes are merged from the *Snapshot*-Branch to the *Trunc*. |
| This automatically triggers the build and the deployment on the |
| Q-environment. |
| |
| <<< |
| |
| == Design decisions |
| |
| All architecture decisions are based on the Architecture Committee Handbook. There are no deviations. |
| |
| <<< |
| |
| == Risks and Technical Debts |
| |
| (Currently there aren't any known issues) |
| |
| <<< |
| |
| == Glossary |
| |
| .Abbreviations and glossary terms |
| [options="header"] |
| |======================================================== |
| |Short|Long|German|Description |
| |AC|Architecture Committee|Architektur-Komittee|Gives framework and constraints according to architecture for oK projects. |
| |CNCU|Central Network Control Unit|| |
| |DAO|Data Access Objects|| |
| |DTO|Data Transfer Object|| |
| |DSO|Distribution System Operator|Verteilnetz-betreiber (VNB)|Manages the distribution network for energy, gas or water. |
| |EPL|Eclipse Public License||Underlying license model for Eclipse projects like contact-base-data@openK |
| |ESB|Enterprise Service Bus||Central instance to exchange data to overcome point-to-point connections. |
| |oK|openKONSEQUENZ|openKONSEQUENZ|Name of the consortium of DSOs |
| |QC|Quality Committee|Qualitätskomitee|Gives framework and constraints according to quality for oK projects. |
| |SCADA|Supervisory Control and Data Acquisition|Netzleitsystem|System, that allows DSOs view/control actual parameters of their power grid. |
| |======================================================== |