Merge pull request #8 from SoftwareAG/feature-migration-1019

Feature migration 1019
Cumulocity-IoT · Feb 14, 2024 · 4dbedf0 · 4dbedf0
2 parents 936182c + 4ed4b32
commit 4dbedf0
Show file tree

Hide file tree

Showing 67 changed files with 18,280 additions and 29,989 deletions.
diff --git a/.github/workflows/check-push.yml b/.github/workflows/check-push.yml
@@ -0,0 +1,26 @@
+name: Check push requests
+
+on:
+  workflow_dispatch:
+  push:
+    # Sequence of patterns matched against refs/tags
+    # tags:
+    # - 'v*' # Push events to matching v*, i.e. v1.0, v20.15.10
+    branches:
+      - '*'
+
+permissions:
+  contents: write
+  issues: write
+  pull-requests: write
+
+jobs:
+  typos:
+    name: Spell Check with Typos
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout Actions Repository
+        uses: actions/checkout@v4
+
+      - name: Check spelling
+        uses: crate-ci/typos@master
diff --git a/.gitignore b/.gitignore
@@ -28,3 +28,4 @@ setEnv.sh
 **/dist
 .env
 attic/
+.angular
diff --git a/.typos.toml b/.typos.toml
@@ -0,0 +1,5 @@
+[default.extend-words]
+Strack = "Strack"
+
+[files]
+extend-exclude = ["**/mappings**.json", "**/*Mappings**.json"]
diff --git a/README.md b/README.md
@@ -1,12 +1,12 @@
 # cumulocity-analytics-management
-> **NOTE**: The plugin can for the moment only be installed in the Administration app, not as shown below direclty in the Streaming Analytics app. This limitation will be fixed shortly!
+
 ## Content
 - [Overview](#overview)
-- [Manage Custom Extension](#manage-custom-extension)
-  - [Upload Custom Extension](#upload-custom-extension)
-  - [Build Custom Extension](#build-custom-extension)
-  - [Options for Custom Extension](#options-for-custom-extension)
-- [Samples Repositories and Building Custom Extensions](#samples-repositories-and-building-custom-extensions)
+- [Manage custom extension](#manage-custom-extension)
+  - [Upload custom extension](#upload-custom-extension)
+  - [Build custom extension](#build-custom-extension)
+  - [Options for custom extension](#options-for-custom-extension)
+- [Samples repositories and building custom extensions](#samples-repositories-and-building-custom-extensions)
 - [Monitoring](#monitoring)
 - [Installation](#installation-of-plugin-as-community-plugin)
 - [Build Instructions](#build-instructions)
@@ -16,9 +16,17 @@
 
 ## Overview
 
-Extends the standard Cumulocity web application with a plugin to manage and add Analytics Builder extensions. Currently the standard UI does not offer the upload of custom blocks via .zip files. This extension enhances the standard Streaming-Analytics UI with these capabilities.
+This solution extends the standard Cumulocity Streaming Analytics application with a plugin to manage and add Analytics Builder extensions. Currently the standard UI does not offer the upload of custom blocks via .zip files. This extension enhances the standard Streaming-Analytics UI with these capabilities.
+
+You can mange the complete lifecycle off an extension:
+* list, upload, download and delete Analytics Builder extensions
+* list installed blocks
+* manage github repositories, which serve as a source for building new extensions on the fly
+* build new extensions from selected blocks and deploy the extensions to the Streaming Analytics engine
+* monitor alarm status and events status send from the Analytics Builder engine, to analyze failures in the deployment process
 
-You can upload blocks that were generated via the [Analytics Builder Block SDK](https://github.com/SoftwareAG/apama-analytics-builder-block-sdk) via the `Add Extension`` button. Drop the .zip file to the modal dialog and the extension will be loaded. In order to use them you have to restart the streaming analytics engine. Click on the "Deploy Extensions (Restart)" button and wait for the notification that the engine was restarted.
+Analytics Builder blocks are build based on the [Analytics Builder Block SDK](https://github.com/SoftwareAG/apama-analytics-builder-block-sdk). Externally build extensions can as well be uploaded via the `Add extension` button. 
+Drop the .zip file to the modal dialog and the extension will be loaded. In order to use them you have to restart the streaming analytics engine. Click on the "Deploy extension (Restart)" button and wait for the notification that the engine was restarted.
 
 ![Extension installation](resources/images/extension-installation-animated.gif)
 
@@ -28,49 +36,49 @@ In addition a table lists all installed analytics blocks with the following info
 
 ![Block list](resources/images/block-list.png)
 
-## Manage Custom Extension
+## Manage custom extension
 Custom extension can be uploaded from your local system. In addition they can be downloaded and deletes as well.
-For a deletion or upload to take effect you need to restart the analytics stremaing engine.
+For a deletion or upload to take effect you need to restart the analytics streaming engine.
 
-### Upload Custom Extension
+### Upload custom extension
 
-After the delployment (restart of streaming analytics) the Block will be available within the Steaming Analytics Application. Deleting a block will remove the block again. Keep in mind that no checking of any usage of that particular custom block is done an thus straming flows might not work anymore.
+After the deployment (restart of streaming analytics) the Block will be available within the Steaming Analytics Application. Deleting a block will remove the block again. Keep in mind that no checking of any usage of that particular custom block is done an thus streaming flows might not work anymore.
 
 ![Use Extension](resources/images/analytics-builder.png)
 
-### Build Custom Extension
-You can build and uploads a custom extension by following the screenflow below:
+### Build custom extension
+You can build and uploads a custom extension by following the screen flow below:
 
-![Build Custom Extensionn](resources/images/create-extension-upload-animated.gif)
+![Build custom extension](resources/images/create-extension-upload-animated.gif)
 
-### Options for Custom Extension
+### Options for custom extension
 For a custom extension you have the following options:
-* Delete: deletes the custom extension permanently. To take effect you need to restart the stremaing analytics engine
-* Details: lists the included blocks of the custom extension on a deail page.
+* Delete: deletes the custom extension permanently. To take effect you need to restart the streaming analytics engine
+* Details: lists the included blocks of the custom extension on a detail page.
 * Download: downloads the custom extension as a zip file.
 
-![Build Custom Extensionn](resources/images/manage-extension.png)
+![Build custom extension](resources/images/manage-extension.png)
 
 
-## Samples Repositories and Building Custom Extensions
-Block samples from github repositories can selected to build custome extension online.
-In order to use this option first you have to configure your Github repsoitory.
-The configured repositories can be updated, deleted and de-/enabled. Only enabled repsoitories are shown in the list of block smaples.
+## Samples repositories and building custom extensions
+Block samples from github repositories can selected to build custom extension online.
+In order to use this option first you have to configure your Github repository.
+The configured repositories can be updated, deleted and de-/enabled. Only enabled repositories are shown in the list of block samples.
 You can manage the github repositories using the following UI:
 
-![Manage Repositories](resources/images/samples-manage-repository.png)
+![Manage repositories](resources/images/samples-manage-repository.png)
 
 In addition you can view the EPL (event processing language) source code:
 
 ![View source code](resources/images/samples-view-code.png)
 
-Building a custom extension starts by selecting the blocks from the list of samples. On selection of blocks an action in the table header appears `Custom Extension`.
+Building a custom extension starts by selecting the blocks from the list of samples. On selection of blocks an action in the table header appears `custom extension`.
 The modal dialog provides the option:
 * to name the custom extension
 * to upload the extension automatically to the Cumulocity Inventory. The created custom extension is otherwise downloaded to locally. It still requires the restart of the analytics engine.
-* to restart the stremaning analytics engine to load the created custom extension.
+* to restart the streaming analytics engine to load the created custom extension.
 
-![Create Extension](resources/images/create-extension-modal.png)
+![Create extension](resources/images/create-extension-modal.png)
 
 ## Monitoring
 
@@ -88,7 +96,7 @@ This solution consists of two parts:
 * the backend as a microservice, which can be found in the release section of the github project: [analytics-ext-service.zip
 ](https://github.com/SoftwareAG/cumulocity-analytics-management/releases)
 
-  If the microservice is not deployed you **can't** build customes extensions.
+  If the microservice is not deployed you **can't** build customs extensions.
 * the web plugin that is installed using the Cumulocity Administration UI (Administration -> Extensions -> Analytics Extensions), see installation below:
 
 The plugin is available as a community plugin and can be installed from the Administration -> Extensions -> Analytics Extensions:
@@ -101,7 +109,7 @@ This guide will teach you how to add the modified Cumulocity standard applicatio
 * to upload the latest plugin release into your tenant, just go to the [Releases](https://github.com/SoftwareAG/cumulocity-analytics-management/releases) and download the analytics-extension.zip package.
 * login to your Cumulocity IoT Tenant, open Administration--Ecosystem--Extensions -> Add extension package
 * upload analytics-extension.zip
-* login to your Cumulocity IoT Tenant and clone Cumulocity standard application in  Administration--Applications--CHOOSE_STANARD_APPLICATION_TO_CONE -> Clone.
+* login to your Cumulocity IoT Tenant and clone Cumulocity standard application in  Administration--Applications--CHOOSE_STANDARD_APPLICATION_TO_CONE -> Clone.
 * to add the modified Cumulocity standard application click on Clone application. After that, select Upload web application and drop the pre-downloaded zip-folder into the field.
 * install plugin in cloned application
 
@@ -111,7 +119,7 @@ Finally, you should see the new application in your App-Switcher.
 **Prerequisites to build plugin:**
 
 * Git 
-* NodeJS (release builds are currently built with `v16.20.0`)
+* NodeJS (release builds are currently built with `v18.19.0`)
 * NPM (Included with NodeJS)
 
 **Instructions**
@@ -135,7 +143,7 @@ git clone https://github.com/SoftwareAG/cumulocity-analytics-management.git
 
 ## Analytics Builder Extension Backend
 
- The microservice downloads the sample blocks from the configured repositories and builds an analytics extension as a zip file. This this zip file is downloaded locally. In an additional step it needs to be uploaded throught UI, see [Upload Custom Extension](#upload-custom-extension).
+ The microservice downloads the sample blocks from the configured repositories and builds an analytics extension as a zip file. This this zip file is downloaded locally. In an additional step it needs to be uploaded through UI, see [Upload custom extension](#upload-custom-extension).
  You can specify if the extension should be uploaded automatically or it should be downloaded by the browser UI.
 
  The microservice is multi tenant ready
@@ -145,7 +153,7 @@ git clone https://github.com/SoftwareAG/cumulocity-analytics-management.git
 * [c8y-go-cli](https://goc8ycli.netlify.app)
 
 ## Local debugging using Vscode/Devcontainer
-To run and debug the microservice locally you an use Vscode and the .devcontainer/devcontainer.json configuration. To test with real c8y microservice credentials create an .env-admin (Administrative Credentials to fetch BootStrap Credentials) file in the analytics service directory and start the container (F1 -> Open in Open Folder in container). The get_service_creds.py fetch the credentials and create the .env file containing the bootstrep credentials. 
+To run and debug the microservice locally you an use Vscode and the .devcontainer/devcontainer.json configuration. To test with real c8y microservice credentials create an .env-admin (Administrative Credentials to fetch BootStrap Credentials) file in the analytics service directory and start the container (F1 -> Open in Open Folder in container). The get_service_creds.py fetch the credentials and create the .env file containing the bootstrap credentials. 
 
 To use your own credentials in the .env file comment the line `"postAttachCommand": "python3 .devcontainer/get_service_creds.py",` in devcontainer.json and create .env file with credentials of your liking. Env file format is:
 
@@ -197,20 +205,15 @@ For the first deployment you have to use:
 # c8y microservices create --file dist/analytics-ext-service.zip --name analytics-ext-service --timeout 360
 ```
 
-<!-- ### Create github access key as tenant option
-
-```
-#  c8y tenantoptions create --category "github" --key "credentials.access_token" --value "XXX" 
-
-``` -->
-## Analytics Builder Block SDK
+## Analytics Builder block SDK
 
 Custom blocks can be generated via the [Analytics Builder block sdk](https://github.com/SoftwareAG/apama-analytics-builder-block-sdk). 
-Find addtional information on how blocks can be developed. However in the release section is one example block included that can be used for test purposes.
+Find additional information on how blocks can be developed. However in the release section is one example block included that can be used for test purposes.
 
 The provided block is an example of the adding the two inputs.
 
 ![Use Extension](resources/images/block-detail.png)
+
 <!-- <br/>
 <p align="center" style="text-indent:70px;">
   <a>
@@ -221,10 +224,10 @@ The provided block is an example of the adding the two inputs.
 
 ## Troubleshooting
 > **Note**
-In order to checkif an extension is deployed look in the log file of the analytics engine for a relevant message:
+In order to check if an extension is deployed look in the log file of the analytics engine for a relevant message:
 `[correlator]  2023-12-04 12:29:43.752 INFO [139659199286272] - Applying extension "/config/extensions/Sample_AB_Extension.zip"`
 
-The log file can be accessed: Adminstration> Ecosystem>Microservices>apama-ctrl-1c-4g>Logs
+The log file can be accessed: Administration> Ecosystem>Microservices>apama-ctrl-1c-4g>Logs
 
 ------------------------------
 

diff --git a/analytics-service/c8y_agent.py b/analytics-service/c8y_agent.py
@@ -48,7 +48,7 @@ def restart_cep(self, request_headers):
                 resource="/service/cep/restart", json={}
             )
         except Exception as e:
-            self._logger.error(f"Ignoring exceptiom!", exc_info=True)
+            self._logger.error(f"Ignoring exception!", exc_info=True)
             # for keys,values in request_headers.items():
             #     self._logger.info(f"Headers: {keys} {values}")
 
@@ -62,40 +62,36 @@ def get_cep_id(self, request_headers):
             ).applications.select(name=self.APAMA_CTRL_APPLICATION_NAME)
             try:
                 app_id = None
+                cep_id = None
                 for app in apps:
                     app_id = app.id
                     self._logger.info(f"Found app id: {app_id}")
                     # break
-                filter = f"$filter=applicationId eq '{app_id}' and name eq '{self.APAMA_CTRL_APPLICATION_NAME}'"
+                query = f"$filter=applicationId eq '{app_id}' and name eq '{self.APAMA_CTRL_APPLICATION_NAME}'"
 
-                self._logger.info(f"Build filter: {filter}")
-                params = {"query": filter}
-                response = self.c8yapp.get_tenant_instance(headers=request_headers).get(
-                    resource=f"/inventory/managedObjects", params=params
-                )
-                cep_id = response["managedObjects"][0]["id"]
-                self._logger.info(f"Found managed object for app: {cep_id}")
-
-                # managed_objects_app = self.c8yapp.get_tenant_instance(
-                #     headers=request_headers
-                # ).inventory.select(query=query)
-                # managed_object_id = None
-                # for managed_object in managed_objects_app:
-                #     self._logger.info(
-                #         f"Found managed object for app: {managed_object}"
-                #     )
-                #     managed_object_id = managed_object
-                #     break
-                # if managed_object_id == None:
-                #     self._logger.error(f"Not found !")
-                #     return None
+                self._logger.info(f"Build filter: {query}")
+
+                managed_objects_app = self.c8yapp.get_tenant_instance(
+                    headers=request_headers
+                ).inventory.select(query)
+                managed_object_id = None
+                for managed_object in managed_objects_app:
+                    self._logger.info(
+                        f"Found managed object: {managed_object.id} for cep app: {app_id}"
+                    )
+                    managed_object_id = managed_object
+                    cep_id = managed_object_id.id
+                    break
+                if managed_object_id == None:
+                    self._logger.error(f"Not found !")
+                    return None
                 return {"id": cep_id}
             except:
                 self._logger.error(
                     f"Error finding app id: {app_id}",
                     exc_info=True,
                 )
         except Exception as e:
-            self._logger.error(f"Ignoring exceptiom!", exc_info=True)
+            self._logger.error(f"Ignoring exception!", exc_info=True)
             # for keys,values in request_headers.items():
             #     self._logger.info(f"Headers: {keys} {values}")
diff --git a/analytics-service/cumulocity.json b/analytics-service/cumulocity.json
@@ -1,6 +1,6 @@
 {
   "apiVersion": "2",
-  "version": "{VERSION}",
+  "version": "{VERSION}-SNAPSHOT",
   "provider": {
     "name": "Software AG"
   },