From dce24ddbd35a52b2c12d223c00daf29041fb90b3 Mon Sep 17 00:00:00 2001 From: Lucas Leblow Date: Fri, 7 Jan 2022 18:10:54 -0700 Subject: [PATCH 01/12] feat: Update pyvolt-dpsim-demo instructions to clarify Signed-off-by: Lucas Leblow --- pyvolt-dpsim-demo/README.md | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/pyvolt-dpsim-demo/README.md b/pyvolt-dpsim-demo/README.md index 7143da0..36dfa65 100644 --- a/pyvolt-dpsim-demo/README.md +++ b/pyvolt-dpsim-demo/README.md @@ -2,6 +2,9 @@ ## Preliminaries +Follow the instructions here to get started: +https://sogno-platform.github.io/docs/getting-started/ + Clone this repo: ```bash git clone https://github.com/sogno-platform/example-deployments.git @@ -22,7 +25,7 @@ helm repo update ``` ### HugePages -The current setup requires HugePages support for the real-time simulator. This can be checked and activated as follows: +The current setup requires HugePages support for the real-time simulator. This can be checked and activated (temporarily) as follows: ```bash # Verify HugePages @@ -54,6 +57,9 @@ HugePages_Surp: 0 Hugepagesize: 2048 kB Hugetlb: 2097152 kB +If you don't see 1024 next to HugePages_Total, you may need to restart +your system and try again with a fresh boot. + # Restart k3s service to apply changes sudo systemctl restart k3s @@ -83,7 +89,7 @@ helm install telegraf influxdata/telegraf -f ts-adapter/telegraf-values.yaml ### Visualization -The following installation will deploy a Grafana instance that is available at the nodePort specified in the grafana_values.yaml file. +The following installation will deploy a Grafana instance that is available at the nodePort specified in the grafana_values.yaml file. Per defautl at port 31230: http://localhost:31230 ```bash @@ -94,11 +100,11 @@ The configmap contains a demo dashboard and should automatically be recognized b ### CIM Editor Pintura -The following installation will deploy a Pintura instance that is available at the nodePort specified in the pintura_values.yaml file. -Per default at port 31234: http://localhost:31234/ +The following installation will deploy a Pintura instance that is available at the nodePort specified in the pintura_values.yaml file. +Per default at port 31234: http://localhost:31234/ (it may take a few moments to launch and become available) ```bash -helm install pintura sogno/pintura -f cim-editor/pintura_values.yaml +helm install pintura sogno/pintura -f cim-editor/pintura_values.yaml ``` ### DPsim Simulation From 04e055d2b79d9efe26e881024e29ff33b2158996 Mon Sep 17 00:00:00 2001 From: Richard Marston Date: Fri, 13 May 2022 16:42:28 +0200 Subject: [PATCH 02/12] Added the first simulation-demo implementation Signed-off-by: Richard Marston --- README.md | 1 + simulation-demo/README.md | 24 +++++++++ simulation-demo/file-service/configmap.yaml | 16 ++++++ simulation-demo/file-service/deploy.sh | 7 +++ simulation-demo/file-service/deployment.yaml | 40 +++++++++++++++ simulation-demo/file-service/restart.sh | 7 +++ simulation-demo/minio/configmap.yaml | 19 ++++++++ simulation-demo/minio/deploy.sh | 51 ++++++++++++++++++++ simulation-demo/minio/deployment.yaml | 27 +++++++++++ simulation-demo/minio/restart.sh | 3 ++ simulation-demo/minio/service.yaml | 14 ++++++ simulation-demo/rabbitmq/deploy.sh | 7 +++ simulation-demo/rabbitmq/deployment.yaml | 22 +++++++++ simulation-demo/rabbitmq/service.yaml | 20 ++++++++ simulation-demo/redis/deploy.sh | 5 ++ simulation-demo/redis/deployment.yaml | 9 ++++ simulation-demo/start_demo.sh | 19 ++++++++ 17 files changed, 291 insertions(+) create mode 100644 simulation-demo/README.md create mode 100644 simulation-demo/file-service/configmap.yaml create mode 100755 simulation-demo/file-service/deploy.sh create mode 100644 simulation-demo/file-service/deployment.yaml create mode 100755 simulation-demo/file-service/restart.sh create mode 100644 simulation-demo/minio/configmap.yaml create mode 100755 simulation-demo/minio/deploy.sh create mode 100644 simulation-demo/minio/deployment.yaml create mode 100755 simulation-demo/minio/restart.sh create mode 100644 simulation-demo/minio/service.yaml create mode 100755 simulation-demo/rabbitmq/deploy.sh create mode 100644 simulation-demo/rabbitmq/deployment.yaml create mode 100644 simulation-demo/rabbitmq/service.yaml create mode 100755 simulation-demo/redis/deploy.sh create mode 100644 simulation-demo/redis/deployment.yaml create mode 100755 simulation-demo/start_demo.sh diff --git a/README.md b/README.md index 4059403..3581e39 100644 --- a/README.md +++ b/README.md @@ -10,3 +10,4 @@ More examples to come soon! - [PMU Data Visualization](pmu-data-visualization) - [Pyvolt DPsim Demo](pyvolt-dpsim-demo) +- [Simulation Demo](simulation-demo) diff --git a/simulation-demo/README.md b/simulation-demo/README.md new file mode 100644 index 0000000..aef9d7a --- /dev/null +++ b/simulation-demo/README.md @@ -0,0 +1,24 @@ +# Simulation Demo + +## Run + +```bash +$ helm repo add sogno https://sogno-platform.github.io/helm-charts +``` + +```bash +$ ./start_demo.sh +``` + +This will call the deploy scripts in each subdirectory, which installs the component from helm chart or deployment file. + +## Notes + + - kubernetes installation required: + * easiest way to achieve this is [with minikube](https://minikube.sigs.k8s.io/docs/start/) + + - security issues (this is only a demo): + * using HTTP instead of HTTPS + * example passwords in yaml files + + diff --git a/simulation-demo/file-service/configmap.yaml b/simulation-demo/file-service/configmap.yaml new file mode 100644 index 0000000..ccb9f05 --- /dev/null +++ b/simulation-demo/file-service/configmap.yaml @@ -0,0 +1,16 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + creationTimestamp: 2016-02-18T19:14:38Z + name: aws-fs-config + namespace: default +data: + credentials: |- + [default] + aws_access_key_id=SECRETUSER + aws_secret_access_key=really/secure/pass + config: |- + [default] + region=us-west-2 + output=json + signature_version = s3v4 diff --git a/simulation-demo/file-service/deploy.sh b/simulation-demo/file-service/deploy.sh new file mode 100755 index 0000000..f076230 --- /dev/null +++ b/simulation-demo/file-service/deploy.sh @@ -0,0 +1,7 @@ +#!/bin/bash + +SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) +pushd $SCRIPT_DIR && echo "Changed to $SCRIPT_DIR" + +kubectl apply -f ./deployment.yaml +kubectl apply -f configmap.yaml diff --git a/simulation-demo/file-service/deployment.yaml b/simulation-demo/file-service/deployment.yaml new file mode 100644 index 0000000..3188286 --- /dev/null +++ b/simulation-demo/file-service/deployment.yaml @@ -0,0 +1,40 @@ +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: sogno-file-service +spec: + selector: + matchLabels: + app: sogno-file-service + template: + metadata: + labels: + app: sogno-file-service + spec: + containers: + - name: sogno-file-service + image: sogno/file-service + ports: + - containerPort: 8080 + volumeMounts: + - name: aws-config-volume + mountPath: /usr/src/app/.aws + volumes: + - name: aws-config-volume + configMap: + name: aws-fs-config + defaultMode: 511 + +--- +apiVersion: v1 +kind: Service +metadata: + name: sogno-file-service +spec: + selector: + app: sogno-file-service + ports: + - name: "8080" + protocol: TCP + port: 8080 diff --git a/simulation-demo/file-service/restart.sh b/simulation-demo/file-service/restart.sh new file mode 100755 index 0000000..2f9c4a9 --- /dev/null +++ b/simulation-demo/file-service/restart.sh @@ -0,0 +1,7 @@ +#!/bin/bash + +SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) +pushd $SCRIPT_DIR && echo "Changed to $SCRIPT_DIR" + +kubectl delete deployment sogno-file-service +./deploy.sh diff --git a/simulation-demo/minio/configmap.yaml b/simulation-demo/minio/configmap.yaml new file mode 100644 index 0000000..a9f5c26 --- /dev/null +++ b/simulation-demo/minio/configmap.yaml @@ -0,0 +1,19 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + creationTimestamp: 2016-02-18T19:14:38Z + name: aws-config + namespace: default +data: + credentials: |- + [default] + aws_access_key_id=SECRETUSER + aws_secret_access_key=really/secure/pass + config: |- + [default] + region=us-west-2 + output=json + signature_version = s3v4 + setup: |- + #!/bin/bash + aws --endpoint-url http://minio:9000 s3 mb s3://sogno-platform diff --git a/simulation-demo/minio/deploy.sh b/simulation-demo/minio/deploy.sh new file mode 100755 index 0000000..e4c5089 --- /dev/null +++ b/simulation-demo/minio/deploy.sh @@ -0,0 +1,51 @@ +#!/bin/bash + +SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) +pushd $SCRIPT_DIR && echo "Changed to $SCRIPT_DIR" + +kubectl apply -f deployment.yaml +kubectl apply -f service.yaml +kubectl apply -f configmap.yaml + +minikube ssh docker pull amazon/aws-cli + +echo "Creating sogno-platform bucket" +kubectl run --rm -i --tty aws-cli --overrides=' +{ + "apiVersion": "v1", + "kind": "Pod", + "metadata": { + "name": "aws-cli" + }, + "spec": { + "containers": [ + { + "name": "aws-cli", + "command": [ "/root/.aws/setup" ], + "spacer": [ "bash" ], + "image": "amazon/aws-cli", + "stdin": true, + "stdinOnce": true, + "tty": true, + "volumeMounts": [ + { + "mountPath": "/root/.aws", + "name": "credentials-volume" + } + ] + } + ], + "volumes": [ + { + "name": "credentials-volume", + "configMap": + { + "name": "aws-config", + "path": "/root/.aws", + "defaultMode": 511 + } + } + ] + } +} +' --image=amazon/aws-cli --restart=Never -- diff --git a/simulation-demo/minio/deployment.yaml b/simulation-demo/minio/deployment.yaml new file mode 100644 index 0000000..ed7b4b5 --- /dev/null +++ b/simulation-demo/minio/deployment.yaml @@ -0,0 +1,27 @@ +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: minio +spec: + selector: + matchLabels: + app: minio + template: + metadata: + labels: + app: minio + spec: + containers: + - name: minio + args: [ "server", "/data", "--console-address", ":9001" ] + command: [ "minio" ] + env: + - name: MINIO_ROOT_USER + value: "SECRETUSER" + - name: MINIO_ROOT_PASSWORD + value: "really/secure/pass" + image: quay.io/minio/minio + ports: + - containerPort: 9000 + - containerPort: 9001 diff --git a/simulation-demo/minio/restart.sh b/simulation-demo/minio/restart.sh new file mode 100755 index 0000000..81f3d7d --- /dev/null +++ b/simulation-demo/minio/restart.sh @@ -0,0 +1,3 @@ +#!/bin/bash +kubectl delete deployment minio +./deploy.sh diff --git a/simulation-demo/minio/service.yaml b/simulation-demo/minio/service.yaml new file mode 100644 index 0000000..af31d88 --- /dev/null +++ b/simulation-demo/minio/service.yaml @@ -0,0 +1,14 @@ +apiVersion: v1 +kind: Service +metadata: + name: minio +spec: + selector: + app: minio + ports: + - name: "9000" + protocol: TCP + port: 9000 + - name: "console" + protocol: TCP + port: 9001 diff --git a/simulation-demo/rabbitmq/deploy.sh b/simulation-demo/rabbitmq/deploy.sh new file mode 100755 index 0000000..b056bb9 --- /dev/null +++ b/simulation-demo/rabbitmq/deploy.sh @@ -0,0 +1,7 @@ +#!/bin/bash + +SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) +pushd $SCRIPT_DIR && echo "Changed to $SCRIPT_DIR" + +kubectl apply -f deployment.yaml +kubectl apply -f service.yaml diff --git a/simulation-demo/rabbitmq/deployment.yaml b/simulation-demo/rabbitmq/deployment.yaml new file mode 100644 index 0000000..851452d --- /dev/null +++ b/simulation-demo/rabbitmq/deployment.yaml @@ -0,0 +1,22 @@ +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: rabbitmq +spec: + selector: + matchLabels: + app: rabbitmq + template: + metadata: + labels: + app: rabbitmq + spec: + containers: + - name: rabbitmq + image: rabbitmq:3 + ports: + - containerPort: 1883 + - containerPort: 5672 + - containerPort: 15672 + - containerPort: 15675 diff --git a/simulation-demo/rabbitmq/service.yaml b/simulation-demo/rabbitmq/service.yaml new file mode 100644 index 0000000..426ec7d --- /dev/null +++ b/simulation-demo/rabbitmq/service.yaml @@ -0,0 +1,20 @@ +apiVersion: v1 +kind: Service +metadata: + name: rabbitmq +spec: + selector: + app: rabbitmq + ports: + - name: "1883" + protocol: TCP + port: 1883 + - name: "5672" + protocol: TCP + port: 5672 + - name: "15672" + protocol: TCP + port: 15672 + - name: "15675" + protocol: TCP + port: 15675 diff --git a/simulation-demo/redis/deploy.sh b/simulation-demo/redis/deploy.sh new file mode 100755 index 0000000..29285b3 --- /dev/null +++ b/simulation-demo/redis/deploy.sh @@ -0,0 +1,5 @@ +#!/bin/bash +#kubectl apply -f deployment.yaml +#helm install redis --set auth.existingSecret=redis-secret bitnami/redis + +helm install redis --set auth.enabled=false bitnami/redis diff --git a/simulation-demo/redis/deployment.yaml b/simulation-demo/redis/deployment.yaml new file mode 100644 index 0000000..14438bb --- /dev/null +++ b/simulation-demo/redis/deployment.yaml @@ -0,0 +1,9 @@ +--- +apiVersion: v1 +kind: Secret +metadata: + name: redis-secret + namespace: default +type: Opaque +data: + redis-password: YWRtaW4xMjM0Cg== diff --git a/simulation-demo/start_demo.sh b/simulation-demo/start_demo.sh new file mode 100755 index 0000000..e194b51 --- /dev/null +++ b/simulation-demo/start_demo.sh @@ -0,0 +1,19 @@ +#!/bin/bash +set -o nounset +set -o errexit + +SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) +pushd $SCRIPT_DIR && echo "Changed to $SCRIPT_DIR" + +echo "Starting rabbitmq" +./rabbitmq/deploy.sh && +echo "Starting redis" && +./redis/deploy.sh && +echo "Starting minio" && +./minio/deploy.sh && +echo "Starting file service" && +./file-service/deploy.sh && +echo "Starting dpsim api" && +helm install dpsim-api sogno/dpsim-api && +echo "Starting dpsim worker" && +helm install dpsim-worker sogno/dpsim-worker From 8695a7bf35678ef6ae90619fb54418386c214e4a Mon Sep 17 00:00:00 2001 From: Federico Paolantoni <1670962+paolantoni@users.noreply.github.com> Date: Fri, 12 Aug 2022 11:23:30 +0200 Subject: [PATCH 03/12] introducing simplified execution of demos with vagrant Signed-off-by: Federico Paolantoni --- README.md | 9 +-- vagrant/Vagrantfile | 41 +++++++++++++ vagrant/pmudatavisualizationdemo.sh | 28 +++++++++ vagrant/pyvoltdpsimdemo.sh | 43 ++++++++++++++ vagrant/simulationdemo.sh | 92 +++++++++++++++++++++++++++++ vagrant/vagrant-basic-bootstrap.sh | 79 +++++++++++++++++++++++++ 6 files changed, 288 insertions(+), 4 deletions(-) create mode 100644 vagrant/Vagrantfile create mode 100644 vagrant/pmudatavisualizationdemo.sh create mode 100644 vagrant/pyvoltdpsimdemo.sh create mode 100644 vagrant/simulationdemo.sh create mode 100644 vagrant/vagrant-basic-bootstrap.sh diff --git a/README.md b/README.md index 3581e39..9a92ad6 100644 --- a/README.md +++ b/README.md @@ -3,11 +3,12 @@ This repository contains deployment instructions and configuration files for different example deployments of the SOGNO platform. More detailed descriptions and high-level architecture descriptions are available on our official [documentation](https://sogno-platform.github.io/docs/) pages. -## Examples +## Running examples using vagrant and virtualbox We currently provide the following examples. More examples to come soon! -- [PMU Data Visualization](pmu-data-visualization) -- [Pyvolt DPsim Demo](pyvolt-dpsim-demo) -- [Simulation Demo](simulation-demo) +- [PMU Data Visualization](pmu-data-visualization) execute: `cd vagrant && vagrant up pmudatavisualization` +- [Pyvolt DPsim Demo](pyvolt-dpsim-demo) execute: `cd vagrant && vagrant up pyvoltdpsim` +- [Simulation Demo](simulation-demo) execute: `cd vagrant && vagrant up simulation` + diff --git a/vagrant/Vagrantfile b/vagrant/Vagrantfile new file mode 100644 index 0000000..4b9483d --- /dev/null +++ b/vagrant/Vagrantfile @@ -0,0 +1,41 @@ +#WARNING: installation will fail in case of air-gapped or proxied systems. + +Vagrant.configure("2") do |config| + config.vm.network "forwarded_port", guest: 31230, host: 31230, protocol: "tcp" + config.vm.network "forwarded_port", guest: 31234, host: 31234, protocol: "tcp" + config.vm.network "forwarded_port", guest: 8080, host: 8080, protocol: "tcp" + config.vm.network "forwarded_port", guest: 6443, host: 6443, protocol: "tcp" #k3s control plane + config.vm.box = "ubuntu/jammy64" ## official canonical image 22.04 lts + #commmon provisioning for all demos + config.vm.provision :shell, path: "vagrant-basic-bootstrap.sh" + #pyvolt-dpsim-demo https://github.com/sogno-platform/example-deployments/tree/main/pyvolt-dpsim-demo + config.vm.define "pyvoltdpsim" do |pyvoltdpsim| + pyvoltdpsim.vm.hostname = "pyvoltdpsim" + pyvoltdpsim.vm.provision :shell, path: "./pyvoltdpsimdemo.sh" + pyvoltdpsim.vm.provider "virtualbox" do |v| + v.name = "pyvoltdpsim" + v.memory = 8192 + v.cpus = 2 + end + end + #simulation-demo https://github.com/sogno-platform/example-deployments/tree/main/simulation-demo + config.vm.define "simulation" do |simulation| + simulation.vm.hostname = "simulation" + simulation.vm.provision :shell, path: "./simulationdemo.sh" + simulation.vm.provider "virtualbox" do |v| + v.name = "simulation" + v.memory = 8192 + v.cpus = 2 + end + end + #pmu data visualization demo https://github.com/sogno-platform/example-deployments/tree/main/pmu-data-visualization + config.vm.define "pmudatavisualization" do |pmudatavisualization| + pmudatavisualization.vm.hostname = "pmudatavisualization" + pmudatavisualization.vm.provision :shell, path: "./pmudatavisualizationdemo.sh" + pmudatavisualization.vm.provider "virtualbox" do |v| + v.name = "pmudatavisualization" + v.memory = 8192 + v.cpus = 2 + end + end +end \ No newline at end of file diff --git a/vagrant/pmudatavisualizationdemo.sh b/vagrant/pmudatavisualizationdemo.sh new file mode 100644 index 0000000..c24b700 --- /dev/null +++ b/vagrant/pmudatavisualizationdemo.sh @@ -0,0 +1,28 @@ +#!/usr/bin/env bash + +cd example-deployments/pmu-data-visualization + +sudo helm install -n sogno-datavis-demo --create-namespace -f databus/rabbitmq_values.yaml rabbitmq bitnami/rabbitmq +sudo helm install influxdb influxdata/influxdb -n sogno-datavis-demo -f ts-database/influxdb-helm-values.yaml +kubectl --namespace sogno-datavis-demo get pods + +#TODO: +#Find the pod +# +#$ kubectl --namespace sogno-datavis-demo get pods +#Login to the pod +# +#$ kubectl --namespace sogno-datavis-demo exec -i -t [pod name] /bin/sh +#Run influxdb CLI +# +#$ influx +#Create database and user telegraf and grant access +# +#> CREATE DATABASE telegraf +#> SHOW Databases +#> CREATE USER telegraf WITH PASSWORD 'telegraf' +#> GRANT ALL ON "telegraf" TO "telegraf" + + +sudo helm install grafana stable/grafana -f visualization/grafana_values.yaml +kubectl get secret -n sogno-datavis-demo grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo \ No newline at end of file diff --git a/vagrant/pyvoltdpsimdemo.sh b/vagrant/pyvoltdpsimdemo.sh new file mode 100644 index 0000000..51a0028 --- /dev/null +++ b/vagrant/pyvoltdpsimdemo.sh @@ -0,0 +1,43 @@ +#!/usr/bin/env bash + +#sogno-demo DATABUS: install rabbitmq via helm +# The `rabbitmq_values.yaml` file contains SOGNO specific overwrites of the default rabbitMQ values. +cat > /etc/rancher/k3s/rabbitmq_values.yaml<< EOF +extraPlugins: rabbitmq_mqtt + +service: + extraPorts: + - name: mqtt + port: 1883 + targetPort: 1883 + nodePort: LoadBalancer + +auth: + username: admin + password: admin +EOF + +RABBITMQISRUNNING=`k3s kubectl --kubeconfig /etc/rancher/k3s/k3s.yaml get pod rabbitmq-0 --output="jsonpath={.status.containerStatuses[*].ready}" | cut -d' ' -f2` +echo "Rabbitmq already running? $RABBITMQISRUNNING"; +if [ ! "$RABBITMQISRUNNING" = true ] ; +then + echo "Installazione di rabbitmq" + sudo helm install -f /etc/rancher/k3s/rabbitmq_values.yaml --kubeconfig /etc/rancher/k3s/k3s.yaml rabbitmq bitnami/rabbitmq +fi + +cd example-deployments/pyvolt-dpsim-demo +# Influx db +sudo helm install influxdb influxdata/influxdb --kubeconfig /etc/rancher/k3s/k3s.yaml -f database/influxdb-helm-values.yaml +# DB adapter +sudo helm install telegraf influxdata/telegraf --kubeconfig /etc/rancher/k3s/k3s.yaml -f ts-adapter/telegraf-values.yaml +# Grafana http://localhost:31230 Username and password for Grafana are set to "demo". +sudo helm install grafana grafana/grafana --kubeconfig /etc/rancher/k3s/k3s.yaml -f visualization/grafana_values.yaml +kubectl apply -f visualization/dashboard-configmap.yaml --kubeconfig /etc/rancher/k3s/k3s.yaml +# Pintura http://localhost:31234 +sudo helm install pintura sogno/pintura --kubeconfig /etc/rancher/k3s/k3s.yaml -f cim-editor/pintura_values.yaml +# DPsim Simulation +sudo helm install dpsim-demo sogno/dpsim-demo --kubeconfig /etc/rancher/k3s/k3s.yaml +# State-Estimation +sudo helm install pyvolt-demo sogno/pyvolt-service --kubeconfig /etc/rancher/k3s/k3s.yaml -f state-estimation/se_values.yaml +echo "Pods running:" +k3s kubectl get pods --all-namespaces -o wide diff --git a/vagrant/simulationdemo.sh b/vagrant/simulationdemo.sh new file mode 100644 index 0000000..a071b30 --- /dev/null +++ b/vagrant/simulationdemo.sh @@ -0,0 +1,92 @@ +#!/usr/bin/env bash + +cd example-deployments/simulation-demo +set -o nounset +set -o errexit + +#SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) +#pushd $SCRIPT_DIR && echo "Changed to $SCRIPT_DIR" + +#cleanup in case of halted vm wich is reprovisioning +k3s kubectl delete deployments --all +k3s kubectl delete services --all +k3s kubectl delete pods --all --grace-period=0 --force +k3s kubectl delete daemonset --all + +echo "Starting rabbitmq" +k3s kubectl apply -f ./rabbitmq/deployment.yaml --kubeconfig /etc/rancher/k3s/k3s.yaml +k3s kubectl apply -f ./rabbitmq/service.yaml --kubeconfig /etc/rancher/k3s/k3s.yaml + +REDISISRUNNING=`k3s kubectl --kubeconfig /etc/rancher/k3s/k3s.yaml get pod redis-master-0 --output="jsonpath={.status.containerStatuses[*].ready}" | cut -d' ' -f2` +echo "Redis already running? $REDISISRUNNING"; +if [ ! "$REDISISRUNNING" = true ] ; +then + echo "installing redis" + sudo helm install redis --set auth.enabled=false bitnami/redis --kubeconfig /etc/rancher/k3s/k3s.yaml + echo "installed redis" +fi + +echo "Starting minio" +k3s kubectl apply -f ./minio/deployment.yaml --kubeconfig /etc/rancher/k3s/k3s.yaml +k3s kubectl apply -f ./minio/service.yaml --kubeconfig /etc/rancher/k3s/k3s.yaml +k3s kubectl apply -f ./minio/configmap.yaml --kubeconfig /etc/rancher/k3s/k3s.yaml + +#installing minikube +#curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube_latest_amd64.deb +#sudo dpkg -i minikube_latest_amd64.deb +#minikube ssh docker pull amazon/aws-cli + +echo "Creating sogno-platform bucket" +k3s kubectl run --kubeconfig /etc/rancher/k3s/k3s.yaml --rm -i --tty aws-cli --overrides=' +{ + "apiVersion": "v1", + "kind": "Pod", + "metadata": { + "name": "aws-cli" + }, + "spec": { + "containers": [ + { + "name": "aws-cli", + "command": [ "/root/.aws/setup" ], + "spacer": [ "bash" ], + "image": "amazon/aws-cli", + "stdin": true, + "stdinOnce": true, + "tty": true, + "volumeMounts": [ + { + "mountPath": "/root/.aws", + "name": "credentials-volume" + } + ] + } + ], + "volumes": [ + { + "name": "credentials-volume", + "configMap": + { + "name": "aws-config", + "path": "/root/.aws", + "defaultMode": 511 + } + } + ] + } +} +' --image=amazon/aws-cli --restart=Never -- + + +echo "Starting file service" && +k3s kubectl apply -f ./file-service/deployment.yaml +k3s kubectl apply -f ./file-service/configmap.yaml + + +echo "Starting dpsim api" && +sudo helm install dpsim-api sogno/dpsim-api --kubeconfig /etc/rancher/k3s/k3s.yaml +echo "Starting dpsim worker" && +sudo helm install dpsim-worker sogno/dpsim-worker --kubeconfig /etc/rancher/k3s/k3s.yaml + +echo "Pods running:" +k3s kubectl get pods --all-namespaces -o wide \ No newline at end of file diff --git a/vagrant/vagrant-basic-bootstrap.sh b/vagrant/vagrant-basic-bootstrap.sh new file mode 100644 index 0000000..0326aa8 --- /dev/null +++ b/vagrant/vagrant-basic-bootstrap.sh @@ -0,0 +1,79 @@ +#!/usr/bin/env bash + +# Increase No of HPgs as required from sogno demo documentation +echo 1024 | sudo tee /proc/sys/vm/nr_hugepages + +sudo ufw disable +apt-get update +apt-get install -y apt-transport-https ca-certificates curl gnupg2 software-properties-common net-tools + + +#installing docker +if ! command -v docker &> /dev/null +then + echo "------- Docker could not be found let's install it -------" + curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - + sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" + sudo apt-get install -y docker-ce + # adding user vagrant to docker group + sudo usermod -aG docker $USER + sudo systemctl enable docker + sudo systemctl start docker +fi + +#installing kubernetes +if ! command -v kubelet &> /dev/null +then + echo "------- Kubernetes could not be found let's install it -------" + curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add - + sudo bash -c 'echo "deb http://apt.kubernetes.io/ kubernetes-xenial main" > /etc/apt/sources.list.d/kubernetes.list' + sudo apt-get update + sudo sudo apt-get install -y kubelet kubeadm kubectl + sudo systemctl enable kubelet + sudo systemctl start kubelet +fi + +##installing helm +if ! command -v helm &> /dev/null +then + echo "------- Helm could not be found let's install it -------" + curl https://baltocdn.com/helm/signing.asc | gpg --dearmor | sudo tee /usr/share/keyrings/helm.gpg > /dev/null + echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/helm.gpg] https://baltocdn.com/helm/stable/debian/ all main" | sudo tee /etc/apt/sources.list.d/helm-stable-debian.list + sudo apt-get update + sudo apt-get install -y helm + sudo helm repo add bitnami https://charts.bitnami.com/bitnami + helm repo add sogno https://sogno-platform.github.io/helm-charts + helm repo add influxdata https://influxdata.github.io/helm-charts + helm repo add grafana https://grafana.github.io/helm-charts + helm repo update +fi +# + +##installing k3s +if ! command -v k3s &> /dev/null +then + echo "------- K3S could not be found let's install it -------" + curl -sfL https://get.k3s.io | sh - + sudo chown -R vagrant /etc/rancher/k3s/ + sudo helm --kubeconfig /etc/rancher/k3s/k3s.yaml list + ##getting pid of installation to wait until completed + PID_K3S_INSTALLATION=$! + export KUBECONFIG=/etc/rancher/k3s/k3s.yaml + #appending variable to bashrc if not exists (to prevent mutliple lines added after repeated provision executions) + grep -qxF 'export KUBECONFIG=/etc/rancher/k3s/k3s.yaml' ~/.bashrc || echo 'export KUBECONFIG=/etc/rancher/k3s/k3s.yaml' >> ~/.bashrc + ##must wait web interface of k3s is available to prevent errors on provisioning + wait $PID_K3S_INSTALLATION +fi + +echo "Pods running:" +k3s kubectl get pods --all-namespaces -o wide + +echo "cluster info" +k3s kubectl cluster-info + +if [ ! -d "example-deployments" ] +then + #checkout examples demo + git clone https://github.com/sogno-platform/example-deployments.git + rm -rf ./example-deployments/vagrant +fi From 92258931909de97ffe8dde91c5a0a240fbdceedb Mon Sep 17 00:00:00 2001 From: Federico Paolantoni <1670962+paolantoni@users.noreply.github.com> Date: Fri, 12 Aug 2022 12:02:34 +0200 Subject: [PATCH 04/12] typo error and updated gitignore. Signed-off-by: Federico Paolantoni --- .gitignore | 1 + vagrant/Vagrantfile | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/.gitignore b/.gitignore index b5d8df1..ebc57b8 100644 --- a/.gitignore +++ b/.gitignore @@ -1,2 +1,3 @@ .vscode *.swp +vagrant/.vagrant/ diff --git a/vagrant/Vagrantfile b/vagrant/Vagrantfile index 4b9483d..3d50051 100644 --- a/vagrant/Vagrantfile +++ b/vagrant/Vagrantfile @@ -4,7 +4,7 @@ Vagrant.configure("2") do |config| config.vm.network "forwarded_port", guest: 31230, host: 31230, protocol: "tcp" config.vm.network "forwarded_port", guest: 31234, host: 31234, protocol: "tcp" config.vm.network "forwarded_port", guest: 8080, host: 8080, protocol: "tcp" - config.vm.network "forwarded_port", guest: 6443, host: 6443, protocol: "tcp" #k3s control plane + config.vm.network "forwarded_port", guest: 6443, host: 6443, protocol: "tcp" #k3s control panel config.vm.box = "ubuntu/jammy64" ## official canonical image 22.04 lts #commmon provisioning for all demos config.vm.provision :shell, path: "vagrant-basic-bootstrap.sh" From 9350026792b1e070915c928173a6e6016bf3b5ea Mon Sep 17 00:00:00 2001 From: Leonardo Falcioni Date: Fri, 12 Aug 2022 14:49:17 +0200 Subject: [PATCH 05/12] new dashboard according to the supposed pattern Signed-off-by: Federico Paolantoni --- .../visualization/dashboard-configmap.yaml | 2914 ++++++++++++++++- 1 file changed, 2754 insertions(+), 160 deletions(-) diff --git a/pyvolt-dpsim-demo/visualization/dashboard-configmap.yaml b/pyvolt-dpsim-demo/visualization/dashboard-configmap.yaml index 9a57430..526940d 100644 --- a/pyvolt-dpsim-demo/visualization/dashboard-configmap.yaml +++ b/pyvolt-dpsim-demo/visualization/dashboard-configmap.yaml @@ -11,69 +11,866 @@ data: "list": [ { "builtIn": 1, - "datasource": "-- Grafana --", + "datasource": { + "type": "datasource", + "uid": "grafana" + }, "enable": true, "hide": true, "iconColor": "rgba(0, 211, 255, 1)", "name": "Annotations & Alerts", + "target": { + "limit": 100, + "matchAny": false, + "tags": [], + "type": "dashboard" + }, "type": "dashboard" } ] }, "editable": true, - "gnetId": null, + "fiscalYearStartMonth": 0, "graphTooltip": 0, + "id": 3, "links": [], + "liveNow": false, "panels": [ { - "aliasColors": {}, - "bars": false, - "dashLength": 10, - "dashes": false, - "datasource": "InfluxDB", + "collapsed": true, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 8 + }, + "id": 9, + "panels": [ + { + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 9, + "w": 24, + "x": 0, + "y": 1 + }, + "id": 4, + "options": { + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom" + }, + "tooltip": { + "mode": "multi", + "sort": "none" + } + }, + "pluginVersion": "9.0.5", + "targets": [ + { + "alias": "Original N1", + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "groupBy": [ + { + "params": [ + "$__interval" + ], + "type": "time" + }, + { + "params": [ + "linear" + ], + "type": "fill" + } + ], + "measurement": "mqtt_consumer", + "orderByTime": "ASC", + "policy": "default", + "queryType": "randomWalk", + "refId": "A", + "resultFormat": "time_series", + "select": [ + [ + { + "params": [ + "data_2" + ], + "type": "field" + }, + { + "params": [], + "type": "last" + }, + { + "params": [ + " / 1000" + ], + "type": "math" + } + ] + ], + "tags": [ + { + "key": "topic", + "operator": "=", + "value": "/dpsim-powerflow" + } + ] + }, + { + "alias": "Estimate N1", + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "groupBy": [ + { + "params": [ + "$__interval" + ], + "type": "time" + }, + { + "params": [ + "linear" + ], + "type": "fill" + } + ], + "hide": false, + "measurement": "mqtt_consumer", + "orderByTime": "ASC", + "policy": "default", + "queryType": "randomWalk", + "refId": "B", + "resultFormat": "time_series", + "select": [ + [ + { + "params": [ + "data_32" + ], + "type": "field" + }, + { + "params": [], + "type": "last" + } + ] + ], + "tags": [ + { + "key": "topic", + "operator": "=", + "value": "/se" + } + ] + } + ], + "title": "Node 1", + "type": "timeseries" + }, + { + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 9, + "w": 24, + "x": 0, + "y": 10 + }, + "id": 7, + "options": { + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom" + }, + "tooltip": { + "mode": "multi", + "sort": "none" + } + }, + "pluginVersion": "9.0.5", + "targets": [ + { + "alias": "Original theta N2", + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "groupBy": [ + { + "params": [ + "$__interval" + ], + "type": "time" + }, + { + "params": [ + "linear" + ], + "type": "fill" + } + ], + "measurement": "mqtt_consumer", + "orderByTime": "ASC", + "policy": "default", + "queryType": "randomWalk", + "refId": "A", + "resultFormat": "time_series", + "select": [ + [ + { + "params": [ + "data_3" + ], + "type": "field" + }, + { + "params": [], + "type": "last" + } + ] + ], + "tags": [ + { + "key": "topic", + "operator": "=", + "value": "/dpsim-powerflow" + } + ] + }, + { + "alias": "Exstimated theta N2", + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "groupBy": [ + { + "params": [ + "$__interval" + ], + "type": "time" + }, + { + "params": [ + "linear" + ], + "type": "fill" + } + ], + "hide": false, + "measurement": "mqtt_consumer", + "orderByTime": "ASC", + "policy": "default", + "queryType": "randomWalk", + "refId": "B", + "resultFormat": "time_series", + "select": [ + [ + { + "params": [ + "data_33" + ], + "type": "field" + }, + { + "params": [], + "type": "last" + } + ] + ], + "tags": [ + { + "key": "topic", + "operator": "=", + "value": "/se" + } + ] + } + ], + "title": "Node 1 - Imag", + "type": "timeseries" + } + ], + "title": "Node 1", + "type": "row" + }, + { + "collapsed": true, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 9 + }, + "id": 6, + "panels": [ + { + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 9, + "w": 24, + "x": 0, + "y": 2 + }, + "id": 2, + "options": { + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom" + }, + "tooltip": { + "mode": "multi", + "sort": "none" + } + }, + "pluginVersion": "9.0.5", + "targets": [ + { + "alias": "Original N2", + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "groupBy": [ + { + "params": [ + "$__interval" + ], + "type": "time" + }, + { + "params": [ + "linear" + ], + "type": "fill" + } + ], + "measurement": "mqtt_consumer", + "orderByTime": "ASC", + "policy": "default", + "queryType": "randomWalk", + "refId": "A", + "resultFormat": "time_series", + "select": [ + [ + { + "params": [ + "data_4" + ], + "type": "field" + }, + { + "params": [], + "type": "last" + }, + { + "params": [ + " / 1000" + ], + "type": "math" + } + ] + ], + "tags": [ + { + "key": "topic", + "operator": "=", + "value": "/dpsim-powerflow" + } + ] + }, + { + "alias": "Estimate N2", + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "groupBy": [ + { + "params": [ + "$__interval" + ], + "type": "time" + }, + { + "params": [ + "linear" + ], + "type": "fill" + } + ], + "hide": false, + "measurement": "mqtt_consumer", + "orderByTime": "ASC", + "policy": "default", + "queryType": "randomWalk", + "refId": "B", + "resultFormat": "time_series", + "select": [ + [ + { + "params": [ + "data_34" + ], + "type": "field" + }, + { + "params": [], + "type": "last" + } + ] + ], + "tags": [ + { + "key": "topic", + "operator": "=", + "value": "/se" + } + ] + } + ], + "title": "Node 2", + "type": "timeseries" + }, + { + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 9, + "w": 24, + "x": 0, + "y": 11 + }, + "id": 10, + "options": { + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom" + }, + "tooltip": { + "mode": "multi", + "sort": "none" + } + }, + "pluginVersion": "9.0.5", + "targets": [ + { + "alias": "Original theta N2", + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "groupBy": [ + { + "params": [ + "$__interval" + ], + "type": "time" + }, + { + "params": [ + "linear" + ], + "type": "fill" + } + ], + "measurement": "mqtt_consumer", + "orderByTime": "ASC", + "policy": "default", + "queryType": "randomWalk", + "refId": "A", + "resultFormat": "time_series", + "select": [ + [ + { + "params": [ + "data_5" + ], + "type": "field" + }, + { + "params": [], + "type": "last" + } + ] + ], + "tags": [ + { + "key": "topic", + "operator": "=", + "value": "/dpsim-powerflow" + } + ] + }, + { + "alias": "Exstimated theta N2", + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "groupBy": [ + { + "params": [ + "$__interval" + ], + "type": "time" + }, + { + "params": [ + "linear" + ], + "type": "fill" + } + ], + "hide": false, + "measurement": "mqtt_consumer", + "orderByTime": "ASC", + "policy": "default", + "queryType": "randomWalk", + "refId": "B", + "resultFormat": "time_series", + "select": [ + [ + { + "params": [ + "data_35" + ], + "type": "field" + }, + { + "params": [], + "type": "last" + } + ] + ], + "tags": [ + { + "key": "topic", + "operator": "=", + "value": "/se" + } + ] + } + ], + "title": "Node 2 - imag", + "type": "timeseries" + } + ], + "title": "Node 2", + "type": "row" + }, + { + "collapsed": false, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 10 + }, + "id": 17, + "panels": [], + "title": "Node 3", + "type": "row" + }, + { + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, "fieldConfig": { "defaults": { - "custom": {} + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "kvolt" }, "overrides": [] }, - "fill": 1, - "fillGradient": 0, "gridPos": { "h": 9, "w": 24, "x": 0, - "y": 0 - }, - "hiddenSeries": false, - "id": 2, - "legend": { - "avg": false, - "current": false, - "max": false, - "min": false, - "show": true, - "total": false, - "values": false - }, - "lines": true, - "linewidth": 1, - "nullPointMode": "null", + "y": 11 + }, + "id": 14, "options": { - "alertThreshold": true - }, - "percentage": false, - "pluginVersion": "7.4.3", - "pointradius": 2, - "points": false, - "renderer": "flot", - "seriesOverrides": [], - "spaceLength": 10, - "stack": false, - "steppedLine": false, + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom" + }, + "tooltip": { + "mode": "multi", + "sort": "none" + } + }, + "pluginVersion": "9.0.5", "targets": [ { - "alias": "Original N2", + "alias": "Original N3", + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, "groupBy": [ { "params": [ @@ -98,7 +895,7 @@ data: [ { "params": [ - "data_4" + "data_6" ], "type": "field" }, @@ -123,7 +920,11 @@ data: ] }, { - "alias": "Estimate N2", + "alias": "Estimate N3", + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, "groupBy": [ { "params": [ @@ -149,7 +950,7 @@ data: [ { "params": [ - "data_34" + "data_36" ], "type": "field" }, @@ -168,96 +969,279 @@ data: ] } ], - "thresholds": [], - "timeFrom": null, - "timeRegions": [], - "timeShift": null, - "title": "Node 2", - "tooltip": { - "shared": true, - "sort": 0, - "value_type": "individual" - }, - "type": "graph", - "xaxis": { - "buckets": null, - "mode": "time", - "name": null, - "show": true, - "values": [] - }, - "yaxes": [ + "title": "Node 3", + "type": "timeseries" + }, + { + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 9, + "w": 24, + "x": 0, + "y": 20 + }, + "id": 15, + "options": { + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom" + }, + "tooltip": { + "mode": "multi", + "sort": "none" + } + }, + "pluginVersion": "9.0.5", + "targets": [ { - "format": "short", - "label": null, - "logBase": 1, - "max": null, - "min": null, - "show": true + "alias": "Original theta N3", + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "groupBy": [ + { + "params": [ + "$__interval" + ], + "type": "time" + }, + { + "params": [ + "linear" + ], + "type": "fill" + } + ], + "measurement": "mqtt_consumer", + "orderByTime": "ASC", + "policy": "default", + "queryType": "randomWalk", + "refId": "A", + "resultFormat": "time_series", + "select": [ + [ + { + "params": [ + "data_7" + ], + "type": "field" + }, + { + "params": [], + "type": "last" + } + ] + ], + "tags": [ + { + "key": "topic", + "operator": "=", + "value": "/dpsim-powerflow" + } + ] }, { - "format": "short", - "label": null, - "logBase": 1, - "max": null, - "min": null, - "show": true + "alias": "Exstimated theta N3", + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "groupBy": [ + { + "params": [ + "$__interval" + ], + "type": "time" + }, + { + "params": [ + "linear" + ], + "type": "fill" + } + ], + "hide": false, + "measurement": "mqtt_consumer", + "orderByTime": "ASC", + "policy": "default", + "queryType": "randomWalk", + "refId": "B", + "resultFormat": "time_series", + "select": [ + [ + { + "params": [ + "data_37" + ], + "type": "field" + }, + { + "params": [], + "type": "last" + } + ] + ], + "tags": [ + { + "key": "topic", + "operator": "=", + "value": "/se" + } + ] } ], - "yaxis": { - "align": false, - "alignLevel": null - } + "title": "Node 3 - imag", + "type": "timeseries" }, { - "aliasColors": {}, - "bars": false, - "dashLength": 10, - "dashes": false, - "datasource": "InfluxDB", + "collapsed": false, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 29 + }, + "id": 19, + "panels": [], + "title": "Node 4", + "type": "row" + }, + { + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, "fieldConfig": { "defaults": { - "custom": {} + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "kvolt" }, "overrides": [] }, - "fill": 1, - "fillGradient": 0, "gridPos": { "h": 9, "w": 24, "x": 0, - "y": 9 + "y": 30 }, - "hiddenSeries": false, - "id": 3, - "legend": { - "avg": false, - "current": false, - "max": false, - "min": false, - "show": true, - "total": false, - "values": false - }, - "lines": true, - "linewidth": 1, - "nullPointMode": "null", + "id": 20, "options": { - "alertThreshold": true - }, - "percentage": false, - "pluginVersion": "7.4.3", - "pointradius": 2, - "points": false, - "renderer": "flot", - "seriesOverrides": [], - "spaceLength": 10, - "stack": false, - "steppedLine": false, + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom" + }, + "tooltip": { + "mode": "multi", + "sort": "none" + } + }, + "pluginVersion": "9.0.5", "targets": [ { - "alias": "Original N11", + "alias": "Original N4", + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, "groupBy": [ { "params": [ @@ -282,7 +1266,7 @@ data: [ { "params": [ - "data_22" + "data_8" ], "type": "field" }, @@ -307,7 +1291,11 @@ data: ] }, { - "alias": "Estimate N11", + "alias": "Estimate N4", + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, "groupBy": [ { "params": [ @@ -333,7 +1321,7 @@ data: [ { "params": [ - "data_44" + "data_38" ], "type": "field" }, @@ -352,62 +1340,1668 @@ data: ] } ], - "thresholds": [], - "timeFrom": null, - "timeRegions": [], - "timeShift": null, - "title": "Node 11", - "tooltip": { - "shared": true, - "sort": 0, - "value_type": "individual" - }, - "type": "graph", - "xaxis": { - "buckets": null, - "mode": "time", - "name": null, - "show": true, - "values": [] - }, - "yaxes": [ - { - "format": "short", - "label": null, - "logBase": 1, - "max": null, - "min": null, - "show": true - }, - { - "format": "short", - "label": null, - "logBase": 1, - "max": null, - "min": null, - "show": true - } - ], - "yaxis": { - "align": false, - "alignLevel": null - } + "title": "Node 4", + "type": "timeseries" + }, + { + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 9, + "w": 24, + "x": 0, + "y": 39 + }, + "id": 21, + "options": { + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom" + }, + "tooltip": { + "mode": "multi", + "sort": "none" + } + }, + "pluginVersion": "9.0.5", + "targets": [ + { + "alias": "Original theta N4", + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "groupBy": [ + { + "params": [ + "$__interval" + ], + "type": "time" + }, + { + "params": [ + "linear" + ], + "type": "fill" + } + ], + "measurement": "mqtt_consumer", + "orderByTime": "ASC", + "policy": "default", + "queryType": "randomWalk", + "refId": "A", + "resultFormat": "time_series", + "select": [ + [ + { + "params": [ + "data_9" + ], + "type": "field" + }, + { + "params": [], + "type": "last" + } + ] + ], + "tags": [ + { + "key": "topic", + "operator": "=", + "value": "/dpsim-powerflow" + } + ] + }, + { + "alias": "Exstimated theta N4", + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "groupBy": [ + { + "params": [ + "$__interval" + ], + "type": "time" + }, + { + "params": [ + "linear" + ], + "type": "fill" + } + ], + "hide": false, + "measurement": "mqtt_consumer", + "orderByTime": "ASC", + "policy": "default", + "queryType": "randomWalk", + "refId": "B", + "resultFormat": "time_series", + "select": [ + [ + { + "params": [ + "data_39" + ], + "type": "field" + }, + { + "params": [], + "type": "last" + } + ] + ], + "tags": [ + { + "key": "topic", + "operator": "=", + "value": "/se" + } + ] + } + ], + "title": "Node 4 - imag", + "type": "timeseries" + }, + { + "collapsed": false, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 48 + }, + "id": 19, + "panels": [], + "title": "Node 5", + "type": "row" + }, + { + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "kvolt" + }, + "overrides": [] + }, + "gridPos": { + "h": 9, + "w": 24, + "x": 0, + "y": 49 + }, + "id": 20, + "options": { + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom" + }, + "tooltip": { + "mode": "multi", + "sort": "none" + } + }, + "pluginVersion": "9.0.5", + "targets": [ + { + "alias": "Original N5", + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "groupBy": [ + { + "params": [ + "$__interval" + ], + "type": "time" + }, + { + "params": [ + "linear" + ], + "type": "fill" + } + ], + "measurement": "mqtt_consumer", + "orderByTime": "ASC", + "policy": "default", + "queryType": "randomWalk", + "refId": "A", + "resultFormat": "time_series", + "select": [ + [ + { + "params": [ + "data_10" + ], + "type": "field" + }, + { + "params": [], + "type": "last" + }, + { + "params": [ + " / 1000" + ], + "type": "math" + } + ] + ], + "tags": [ + { + "key": "topic", + "operator": "=", + "value": "/dpsim-powerflow" + } + ] + }, + { + "alias": "Estimate N5", + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "groupBy": [ + { + "params": [ + "$__interval" + ], + "type": "time" + }, + { + "params": [ + "linear" + ], + "type": "fill" + } + ], + "hide": false, + "measurement": "mqtt_consumer", + "orderByTime": "ASC", + "policy": "default", + "queryType": "randomWalk", + "refId": "B", + "resultFormat": "time_series", + "select": [ + [ + { + "params": [ + "data_40" + ], + "type": "field" + }, + { + "params": [], + "type": "last" + } + ] + ], + "tags": [ + { + "key": "topic", + "operator": "=", + "value": "/se" + } + ] + } + ], + "title": "Node 5", + "type": "timeseries" + }, + { + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 9, + "w": 24, + "x": 0, + "y": 58 + }, + "id": 21, + "options": { + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom" + }, + "tooltip": { + "mode": "multi", + "sort": "none" + } + }, + "pluginVersion": "9.0.5", + "targets": [ + { + "alias": "Original theta N5", + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "groupBy": [ + { + "params": [ + "$__interval" + ], + "type": "time" + }, + { + "params": [ + "linear" + ], + "type": "fill" + } + ], + "measurement": "mqtt_consumer", + "orderByTime": "ASC", + "policy": "default", + "queryType": "randomWalk", + "refId": "A", + "resultFormat": "time_series", + "select": [ + [ + { + "params": [ + "data_11" + ], + "type": "field" + }, + { + "params": [], + "type": "last" + } + ] + ], + "tags": [ + { + "key": "topic", + "operator": "=", + "value": "/dpsim-powerflow" + } + ] + }, + { + "alias": "Estimated theta N5", + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "groupBy": [ + { + "params": [ + "$__interval" + ], + "type": "time" + }, + { + "params": [ + "linear" + ], + "type": "fill" + } + ], + "hide": false, + "measurement": "mqtt_consumer", + "orderByTime": "ASC", + "policy": "default", + "queryType": "randomWalk", + "refId": "B", + "resultFormat": "time_series", + "select": [ + [ + { + "params": [ + "data_41" + ], + "type": "field" + }, + { + "params": [], + "type": "last" + } + ] + ], + "tags": [ + { + "key": "topic", + "operator": "=", + "value": "/se" + } + ] + } + ], + "title": "Node 5 - imag", + "type": "timeseries" + }, + { + "collapsed": false, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 67 + }, + "id": 19, + "panels": [], + "title": "Node 6", + "type": "row" + }, + { + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "kvolt" + }, + "overrides": [] + }, + "gridPos": { + "h": 9, + "w": 24, + "x": 0, + "y": 68 + }, + "id": 20, + "options": { + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom" + }, + "tooltip": { + "mode": "multi", + "sort": "none" + } + }, + "pluginVersion": "9.0.5", + "targets": [ + { + "alias": "Original N6", + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "groupBy": [ + { + "params": [ + "$__interval" + ], + "type": "time" + }, + { + "params": [ + "linear" + ], + "type": "fill" + } + ], + "measurement": "mqtt_consumer", + "orderByTime": "ASC", + "policy": "default", + "queryType": "randomWalk", + "refId": "A", + "resultFormat": "time_series", + "select": [ + [ + { + "params": [ + "data_12" + ], + "type": "field" + }, + { + "params": [], + "type": "last" + }, + { + "params": [ + " / 1000" + ], + "type": "math" + } + ] + ], + "tags": [ + { + "key": "topic", + "operator": "=", + "value": "/dpsim-powerflow" + } + ] + }, + { + "alias": "Estimate N6", + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "groupBy": [ + { + "params": [ + "$__interval" + ], + "type": "time" + }, + { + "params": [ + "linear" + ], + "type": "fill" + } + ], + "hide": false, + "measurement": "mqtt_consumer", + "orderByTime": "ASC", + "policy": "default", + "queryType": "randomWalk", + "refId": "B", + "resultFormat": "time_series", + "select": [ + [ + { + "params": [ + "data_42" + ], + "type": "field" + }, + { + "params": [], + "type": "last" + } + ] + ], + "tags": [ + { + "key": "topic", + "operator": "=", + "value": "/se" + } + ] + } + ], + "title": "Node 6", + "type": "timeseries" + }, + { + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 9, + "w": 24, + "x": 0, + "y": 77 + }, + "id": 21, + "options": { + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom" + }, + "tooltip": { + "mode": "multi", + "sort": "none" + } + }, + "pluginVersion": "9.0.5", + "targets": [ + { + "alias": "Original theta N6", + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "groupBy": [ + { + "params": [ + "$__interval" + ], + "type": "time" + }, + { + "params": [ + "linear" + ], + "type": "fill" + } + ], + "measurement": "mqtt_consumer", + "orderByTime": "ASC", + "policy": "default", + "queryType": "randomWalk", + "refId": "A", + "resultFormat": "time_series", + "select": [ + [ + { + "params": [ + "data_13" + ], + "type": "field" + }, + { + "params": [], + "type": "last" + } + ] + ], + "tags": [ + { + "key": "topic", + "operator": "=", + "value": "/dpsim-powerflow" + } + ] + }, + { + "alias": "Estimated theta N6", + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "groupBy": [ + { + "params": [ + "$__interval" + ], + "type": "time" + }, + { + "params": [ + "linear" + ], + "type": "fill" + } + ], + "hide": false, + "measurement": "mqtt_consumer", + "orderByTime": "ASC", + "policy": "default", + "queryType": "randomWalk", + "refId": "B", + "resultFormat": "time_series", + "select": [ + [ + { + "params": [ + "data_43" + ], + "type": "field" + }, + { + "params": [], + "type": "last" + } + ] + ], + "tags": [ + { + "key": "topic", + "operator": "=", + "value": "/se" + } + ] + } + ], + "title": "Node 6 - imag", + "type": "timeseries" + }, + { + "collapsed": true, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 86 + }, + "id": 12, + "panels": [ + { + "aliasColors": {}, + "bars": false, + "dashLength": 10, + "dashes": false, + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "fill": 1, + "fillGradient": 0, + "gridPos": { + "h": 9, + "w": 24, + "x": 0, + "y": 5 + }, + "hiddenSeries": false, + "id": 3, + "legend": { + "avg": false, + "current": false, + "max": false, + "min": false, + "show": true, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 1, + "nullPointMode": "null", + "options": { + "alertThreshold": true + }, + "percentage": false, + "pluginVersion": "9.0.5", + "pointradius": 2, + "points": false, + "renderer": "flot", + "seriesOverrides": [], + "spaceLength": 10, + "stack": false, + "steppedLine": false, + "targets": [ + { + "alias": "Original N11", + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "groupBy": [ + { + "params": [ + "$__interval" + ], + "type": "time" + }, + { + "params": [ + "linear" + ], + "type": "fill" + } + ], + "measurement": "mqtt_consumer", + "orderByTime": "ASC", + "policy": "default", + "queryType": "randomWalk", + "refId": "A", + "resultFormat": "time_series", + "select": [ + [ + { + "params": [ + "data_22" + ], + "type": "field" + }, + { + "params": [], + "type": "last" + }, + { + "params": [ + " / 1000" + ], + "type": "math" + } + ] + ], + "tags": [ + { + "key": "topic", + "operator": "=", + "value": "/dpsim-powerflow" + } + ] + }, + { + "alias": "Estimate N11", + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "groupBy": [ + { + "params": [ + "$__interval" + ], + "type": "time" + }, + { + "params": [ + "linear" + ], + "type": "fill" + } + ], + "hide": false, + "measurement": "mqtt_consumer", + "orderByTime": "ASC", + "policy": "default", + "queryType": "randomWalk", + "refId": "B", + "resultFormat": "time_series", + "select": [ + [ + { + "params": [ + "data_52" + ], + "type": "field" + }, + { + "params": [], + "type": "last" + } + ] + ], + "tags": [ + { + "key": "topic", + "operator": "=", + "value": "/se" + } + ] + } + ], + "thresholds": [], + "timeRegions": [], + "title": "Node 11", + "tooltip": { + "shared": true, + "sort": 0, + "value_type": "individual" + }, + "type": "graph", + "xaxis": { + "mode": "time", + "show": true, + "values": [] + }, + "yaxes": [ + { + "format": "short", + "logBase": 1, + "show": true + }, + { + "format": "short", + "logBase": 1, + "show": true + } + ], + "yaxis": { + "align": false + } + }, + { + "aliasColors": {}, + "bars": false, + "dashLength": 10, + "dashes": false, + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "fill": 1, + "fillGradient": 0, + "gridPos": { + "h": 9, + "w": 24, + "x": 0, + "y": 14 + }, + "hiddenSeries": false, + "id": 13, + "legend": { + "avg": false, + "current": false, + "max": false, + "min": false, + "show": true, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 1, + "nullPointMode": "null", + "options": { + "alertThreshold": true + }, + "percentage": false, + "pluginVersion": "9.0.5", + "pointradius": 2, + "points": false, + "renderer": "flot", + "seriesOverrides": [], + "spaceLength": 10, + "stack": false, + "steppedLine": false, + "targets": [ + { + "alias": "Original N11", + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "groupBy": [ + { + "params": [ + "$__interval" + ], + "type": "time" + }, + { + "params": [ + "linear" + ], + "type": "fill" + } + ], + "measurement": "mqtt_consumer", + "orderByTime": "ASC", + "policy": "default", + "queryType": "randomWalk", + "refId": "A", + "resultFormat": "time_series", + "select": [ + [ + { + "params": [ + "data_23" + ], + "type": "field" + }, + { + "params": [], + "type": "last" + } + ] + ], + "tags": [ + { + "key": "topic", + "operator": "=", + "value": "/dpsim-powerflow" + } + ] + }, + { + "alias": "Estimate N11", + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "groupBy": [ + { + "params": [ + "$__interval" + ], + "type": "time" + }, + { + "params": [ + "linear" + ], + "type": "fill" + } + ], + "hide": false, + "measurement": "mqtt_consumer", + "orderByTime": "ASC", + "policy": "default", + "queryType": "randomWalk", + "refId": "B", + "resultFormat": "time_series", + "select": [ + [ + { + "params": [ + "data_53" + ], + "type": "field" + }, + { + "params": [], + "type": "last" + } + ] + ], + "tags": [ + { + "key": "topic", + "operator": "=", + "value": "/se" + } + ] + } + ], + "thresholds": [], + "timeRegions": [], + "title": "Node 12", + "tooltip": { + "shared": true, + "sort": 0, + "value_type": "individual" + }, + "type": "graph", + "xaxis": { + "mode": "time", + "show": true, + "values": [] + }, + "yaxes": [ + { + "format": "short", + "logBase": 1, + "show": true + }, + { + "format": "short", + "logBase": 1, + "show": true + } + ], + "yaxis": { + "align": false + } + } + ], + "title": "Node 11", + "type": "row" + }, + { + "collapsed": true, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 87 + }, + "id": 12, + "panels": [ + { + "aliasColors": {}, + "bars": false, + "dashLength": 10, + "dashes": false, + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "fill": 1, + "fillGradient": 0, + "gridPos": { + "h": 9, + "w": 24, + "x": 0, + "y": 5 + }, + "hiddenSeries": false, + "id": 3, + "legend": { + "avg": false, + "current": false, + "max": false, + "min": false, + "show": true, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 1, + "nullPointMode": "null", + "options": { + "alertThreshold": true + }, + "percentage": false, + "pluginVersion": "9.0.5", + "pointradius": 2, + "points": false, + "renderer": "flot", + "seriesOverrides": [], + "spaceLength": 10, + "stack": false, + "steppedLine": false, + "targets": [ + { + "alias": "Original N12", + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "groupBy": [ + { + "params": [ + "$__interval" + ], + "type": "time" + }, + { + "params": [ + "linear" + ], + "type": "fill" + } + ], + "measurement": "mqtt_consumer", + "orderByTime": "ASC", + "policy": "default", + "queryType": "randomWalk", + "refId": "A", + "resultFormat": "time_series", + "select": [ + [ + { + "params": [ + "data_24" + ], + "type": "field" + }, + { + "params": [], + "type": "last" + }, + { + "params": [ + " / 1000" + ], + "type": "math" + } + ] + ], + "tags": [ + { + "key": "topic", + "operator": "=", + "value": "/dpsim-powerflow" + } + ] + }, + { + "alias": "Estimate N12", + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "groupBy": [ + { + "params": [ + "$__interval" + ], + "type": "time" + }, + { + "params": [ + "linear" + ], + "type": "fill" + } + ], + "hide": false, + "measurement": "mqtt_consumer", + "orderByTime": "ASC", + "policy": "default", + "queryType": "randomWalk", + "refId": "B", + "resultFormat": "time_series", + "select": [ + [ + { + "params": [ + "data_54" + ], + "type": "field" + }, + { + "params": [], + "type": "last" + } + ] + ], + "tags": [ + { + "key": "topic", + "operator": "=", + "value": "/se" + } + ] + } + ], + "thresholds": [], + "timeRegions": [], + "title": "Node 12", + "tooltip": { + "shared": true, + "sort": 0, + "value_type": "individual" + }, + "type": "graph", + "xaxis": { + "mode": "time", + "show": true, + "values": [] + }, + "yaxes": [ + { + "format": "short", + "logBase": 1, + "show": true + }, + { + "format": "short", + "logBase": 1, + "show": true + } + ], + "yaxis": { + "align": false + } + }, + { + "aliasColors": {}, + "bars": false, + "dashLength": 10, + "dashes": false, + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "fill": 1, + "fillGradient": 0, + "gridPos": { + "h": 9, + "w": 24, + "x": 0, + "y": 14 + }, + "hiddenSeries": false, + "id": 13, + "legend": { + "avg": false, + "current": false, + "max": false, + "min": false, + "show": true, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 1, + "nullPointMode": "null", + "options": { + "alertThreshold": true + }, + "percentage": false, + "pluginVersion": "9.0.5", + "pointradius": 2, + "points": false, + "renderer": "flot", + "seriesOverrides": [], + "spaceLength": 10, + "stack": false, + "steppedLine": false, + "targets": [ + { + "alias": "Original N12", + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "groupBy": [ + { + "params": [ + "$__interval" + ], + "type": "time" + }, + { + "params": [ + "linear" + ], + "type": "fill" + } + ], + "measurement": "mqtt_consumer", + "orderByTime": "ASC", + "policy": "default", + "queryType": "randomWalk", + "refId": "A", + "resultFormat": "time_series", + "select": [ + [ + { + "params": [ + "data_25" + ], + "type": "field" + }, + { + "params": [], + "type": "last" + } + ] + ], + "tags": [ + { + "key": "topic", + "operator": "=", + "value": "/dpsim-powerflow" + } + ] + }, + { + "alias": "Estimate N12", + "datasource": { + "type": "influxdb", + "uid": "P951FEA4DE68E13C5" + }, + "groupBy": [ + { + "params": [ + "$__interval" + ], + "type": "time" + }, + { + "params": [ + "linear" + ], + "type": "fill" + } + ], + "hide": false, + "measurement": "mqtt_consumer", + "orderByTime": "ASC", + "policy": "default", + "queryType": "randomWalk", + "refId": "B", + "resultFormat": "time_series", + "select": [ + [ + { + "params": [ + "data_55" + ], + "type": "field" + }, + { + "params": [], + "type": "last" + } + ] + ], + "tags": [ + { + "key": "topic", + "operator": "=", + "value": "/se" + } + ] + } + ], + "thresholds": [], + "timeRegions": [], + "title": "Node 12", + "tooltip": { + "shared": true, + "sort": 0, + "value_type": "individual" + }, + "type": "graph", + "xaxis": { + "mode": "time", + "show": true, + "values": [] + }, + "yaxes": [ + { + "format": "short", + "logBase": 1, + "show": true + }, + { + "format": "short", + "logBase": 1, + "show": true + } + ], + "yaxis": { + "align": false + } + } + ], + "title": "Node 12", + "type": "row" } ], "refresh": false, - "schemaVersion": 27, + "schemaVersion": 36, "style": "dark", "tags": [], "templating": { "list": [] }, "time": { - "from": "2021-03-25T11:42:34.842Z", - "to": "2021-03-25T11:45:15.486Z" + "from": "now-5m", + "to": "now" }, "timepicker": {}, "timezone": "", - "title": "Pyvolt-DPsim-Demo", - "uid": "eOcXX7wGk", - "version": 1 + "title": "Pyvolt-DPsim-Demo-new Copy", + "uid": "qHCj-KiVk", + "version": 1, + "weekStart": "" } From 3ff3c126b24bc776bda1c1506fe43c9b8331d4a9 Mon Sep 17 00:00:00 2001 From: Leonardo Falcioni Date: Fri, 12 Aug 2022 16:50:37 +0200 Subject: [PATCH 06/12] provisioning for vmware_workstation provider added Signed-off-by: Leonardo Falcioni --- vagrant/Vagrantfile | 3 +++ 1 file changed, 3 insertions(+) diff --git a/vagrant/Vagrantfile b/vagrant/Vagrantfile index 3d50051..43f6705 100644 --- a/vagrant/Vagrantfile +++ b/vagrant/Vagrantfile @@ -8,6 +8,9 @@ Vagrant.configure("2") do |config| config.vm.box = "ubuntu/jammy64" ## official canonical image 22.04 lts #commmon provisioning for all demos config.vm.provision :shell, path: "vagrant-basic-bootstrap.sh" + config.vm.provider "vmware_workstation" do |v, override| + override.vm.box = "generic/ubuntu1804" + end #pyvolt-dpsim-demo https://github.com/sogno-platform/example-deployments/tree/main/pyvolt-dpsim-demo config.vm.define "pyvoltdpsim" do |pyvoltdpsim| pyvoltdpsim.vm.hostname = "pyvoltdpsim" From 8b92a2dc040b98416e74a946051223eb2b5eaaea Mon Sep 17 00:00:00 2001 From: Leonardo Falcioni Date: Fri, 12 Aug 2022 16:52:03 +0200 Subject: [PATCH 07/12] pmudatavisualization demo script updated Signed-off-by: Leonardo Falcioni --- vagrant/pmudatavisualizationdemo.sh | 85 +++++++++++++++++++++-------- 1 file changed, 61 insertions(+), 24 deletions(-) diff --git a/vagrant/pmudatavisualizationdemo.sh b/vagrant/pmudatavisualizationdemo.sh index c24b700..c7edb97 100644 --- a/vagrant/pmudatavisualizationdemo.sh +++ b/vagrant/pmudatavisualizationdemo.sh @@ -1,28 +1,65 @@ #!/usr/bin/env bash +#cleanup in case of halted vm wich is reprovisioning +k3s kubectl delete deployments --all +k3s kubectl delete services --all +k3s kubectl delete pods --all --grace-period=0 --force +k3s kubectl delete daemonset --all + +# some necessary fixes +cat > example-deployments/pmu-data-visualization/ts-adapter/telegraf.conf<< EOF +[[outputs.influxdb]] + urls = ["http://influxdb:8086"] + database = "telegraf" + skip_database_creation = false + username = "telegraf" + password = "telegraf" + retention_policy = "" + write_consistency = "any" + timeout = "5s" + +[[inputs.mqtt_consumer]] + servers = ["tcp://rabbitmq:1883"] + + ## Topics that will be subscribed to. + topics = [ + "/pmu/dev", + "/pmu/field" + ] + + ## Username and password to connect MQTT server. + username = "admin" + password = "admin" + + data_format = "json_struct" + json_struct_name = "sogno-device" +EOF + + +cp -f example-deployments/pyvolt-dpsim-demo/visualization/grafana_values.yaml example-deployments/pmu-data-visualization/visualization/ + cd example-deployments/pmu-data-visualization -sudo helm install -n sogno-datavis-demo --create-namespace -f databus/rabbitmq_values.yaml rabbitmq bitnami/rabbitmq -sudo helm install influxdb influxdata/influxdb -n sogno-datavis-demo -f ts-database/influxdb-helm-values.yaml -kubectl --namespace sogno-datavis-demo get pods - -#TODO: -#Find the pod -# -#$ kubectl --namespace sogno-datavis-demo get pods -#Login to the pod -# -#$ kubectl --namespace sogno-datavis-demo exec -i -t [pod name] /bin/sh -#Run influxdb CLI -# -#$ influx -#Create database and user telegraf and grant access -# -#> CREATE DATABASE telegraf -#> SHOW Databases -#> CREATE USER telegraf WITH PASSWORD 'telegraf' -#> GRANT ALL ON "telegraf" TO "telegraf" - - -sudo helm install grafana stable/grafana -f visualization/grafana_values.yaml -kubectl get secret -n sogno-datavis-demo grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo \ No newline at end of file +# rabbitMQ +sudo helm install rabbitmq bitnami/rabbitmq --kubeconfig /etc/rancher/k3s/k3s.yaml -f databus/rabbitmq_values.yaml +# Influx db +sudo helm install influxdb influxdata/influxdb --kubeconfig /etc/rancher/k3s/k3s.yaml -f database/influxdb-helm-values.yaml +# Grafana +sudo helm install grafana grafana/grafana --kubeconfig /etc/rancher/k3s/k3s.yaml -f visualization/grafana_values.yaml + +#patched version of Telegraf as a database adapter, we deploy the adapter in conjunction with the created config map +k3s kubectl --kubeconfig /etc/rancher/k3s/k3s.yaml apply -k ts-adapter/ +k3s kubectl --kubeconfig /etc/rancher/k3s/k3s.yaml apply -f ts-adapter/telegraf-deployment.yaml + +# wait until all the pods are running +k3s kubectl wait --for=condition=Ready pods --all +# start manually telegraf into the pod with the patched telegraf version +TELEGRAFPODNAME=$(k3s kubectl get pods | grep telegraf | awk '{print $1}') +k3s kubectl wait --for=condition=Running pod $TELEGRAFPODNAME +# relaunch it detached (started previously in the image rwthacs/telegraf-sogno ) +k3s kubectl --kubeconfig /etc/rancher/k3s/k3s.yaml exec $TELEGRAFPODNAME -- bash -c './go/bin/telegraf'> /dev/null 2>&1 & + +echo "Pods running:" +k3s kubectl --kubeconfig /etc/rancher/k3s/k3s.yaml get pods -o wide +echo "Password for Web UI:" +k3s kubectl --kubeconfig /etc/rancher/k3s/k3s.yaml get secret grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo \ No newline at end of file From 2e4836669b1b221334dda959fd73b863f4626c74 Mon Sep 17 00:00:00 2001 From: Federico Paolantoni <1670962+paolantoni@users.noreply.github.com> Date: Wed, 17 Aug 2022 16:04:33 +0200 Subject: [PATCH 08/12] k3s installaiton moved to specific provisioning scripts because of minikube usage in simulationdemo Signed-off-by: Federico Paolantoni --- vagrant/Vagrantfile | 87 +++++------ vagrant/pmudatavisualizationdemo.sh | 18 +++ vagrant/pyvoltdpsimdemo.sh | 18 +++ vagrant/simulationdemo.sh | 215 ++++++++++++++++------------ vagrant/vagrant-basic-bootstrap.sh | 136 ++++++++---------- 5 files changed, 260 insertions(+), 214 deletions(-) diff --git a/vagrant/Vagrantfile b/vagrant/Vagrantfile index 43f6705..76b7a56 100644 --- a/vagrant/Vagrantfile +++ b/vagrant/Vagrantfile @@ -1,44 +1,45 @@ -#WARNING: installation will fail in case of air-gapped or proxied systems. - -Vagrant.configure("2") do |config| - config.vm.network "forwarded_port", guest: 31230, host: 31230, protocol: "tcp" - config.vm.network "forwarded_port", guest: 31234, host: 31234, protocol: "tcp" - config.vm.network "forwarded_port", guest: 8080, host: 8080, protocol: "tcp" - config.vm.network "forwarded_port", guest: 6443, host: 6443, protocol: "tcp" #k3s control panel - config.vm.box = "ubuntu/jammy64" ## official canonical image 22.04 lts - #commmon provisioning for all demos - config.vm.provision :shell, path: "vagrant-basic-bootstrap.sh" - config.vm.provider "vmware_workstation" do |v, override| - override.vm.box = "generic/ubuntu1804" - end - #pyvolt-dpsim-demo https://github.com/sogno-platform/example-deployments/tree/main/pyvolt-dpsim-demo - config.vm.define "pyvoltdpsim" do |pyvoltdpsim| - pyvoltdpsim.vm.hostname = "pyvoltdpsim" - pyvoltdpsim.vm.provision :shell, path: "./pyvoltdpsimdemo.sh" - pyvoltdpsim.vm.provider "virtualbox" do |v| - v.name = "pyvoltdpsim" - v.memory = 8192 - v.cpus = 2 - end - end - #simulation-demo https://github.com/sogno-platform/example-deployments/tree/main/simulation-demo - config.vm.define "simulation" do |simulation| - simulation.vm.hostname = "simulation" - simulation.vm.provision :shell, path: "./simulationdemo.sh" - simulation.vm.provider "virtualbox" do |v| - v.name = "simulation" - v.memory = 8192 - v.cpus = 2 - end - end - #pmu data visualization demo https://github.com/sogno-platform/example-deployments/tree/main/pmu-data-visualization - config.vm.define "pmudatavisualization" do |pmudatavisualization| - pmudatavisualization.vm.hostname = "pmudatavisualization" - pmudatavisualization.vm.provision :shell, path: "./pmudatavisualizationdemo.sh" - pmudatavisualization.vm.provider "virtualbox" do |v| - v.name = "pmudatavisualization" - v.memory = 8192 - v.cpus = 2 - end - end +#WARNING: installation will fail in case of air-gapped or proxied systems. + +Vagrant.configure("2") do |config| + config.vm.network "forwarded_port", guest: 31230, host: 31230, protocol: "tcp" + config.vm.network "forwarded_port", guest: 31234, host: 31234, protocol: "tcp" + config.vm.network "forwarded_port", guest: 8080, host: 8080, protocol: "tcp" + config.vm.network "forwarded_port", guest: 6443, host: 6443, protocol: "tcp" #k3s control panel + config.vm.box = "ubuntu/jammy64" ## official canonical image 22.04 lts + + #commmon provisioning for all demos + config.vm.provision :shell, path: "vagrant-basic-bootstrap.sh" + config.vm.provider "vmware_workstation" do |v, override| + override.vm.box = "generic/ubuntu1804" + end + #pyvolt-dpsim-demo https://github.com/sogno-platform/example-deployments/tree/main/pyvolt-dpsim-demo + config.vm.define "pyvoltdpsim" do |pyvoltdpsim| + pyvoltdpsim.vm.hostname = "pyvoltdpsim" + pyvoltdpsim.vm.provision :shell, path: "./pyvoltdpsimdemo.sh" + pyvoltdpsim.vm.provider "virtualbox" do |v| + v.name = "pyvoltdpsim" + v.memory = 8192 + v.cpus = 2 + end + end + #simulation-demo https://github.com/sogno-platform/example-deployments/tree/main/simulation-demo + config.vm.define "simulation" do |simulation| + simulation.vm.hostname = "simulation" + simulation.vm.provision :shell, path: "./simulationdemo.sh" + simulation.vm.provider "virtualbox" do |v| + v.name = "simulation" + v.memory = 8192 + v.cpus = 2 + end + end + #pmu data visualization demo https://github.com/sogno-platform/example-deployments/tree/main/pmu-data-visualization + config.vm.define "pmudatavisualization" do |pmudatavisualization| + pmudatavisualization.vm.hostname = "pmudatavisualization" + pmudatavisualization.vm.provision :shell, path: "./pmudatavisualizationdemo.sh" + pmudatavisualization.vm.provider "virtualbox" do |v| + v.name = "pmudatavisualization" + v.memory = 8192 + v.cpus = 2 + end + end end \ No newline at end of file diff --git a/vagrant/pmudatavisualizationdemo.sh b/vagrant/pmudatavisualizationdemo.sh index c7edb97..83c29e8 100644 --- a/vagrant/pmudatavisualizationdemo.sh +++ b/vagrant/pmudatavisualizationdemo.sh @@ -1,5 +1,23 @@ #!/usr/bin/env bash +#installing k3s +if ! command -v k3s &> /dev/null +then + echo "------- K3S could not be found let's install it -------" + curl -sfL https://get.k3s.io | sh - + sudo chown -R vagrant /etc/rancher/k3s/ + sudo helm --kubeconfig /etc/rancher/k3s/k3s.yaml list + ##getting pid of installation to wait until completed + PID_K3S_INSTALLATION=$! + export KUBECONFIG=/etc/rancher/k3s/k3s.yaml + #appending variable to bashrc if not exists (to prevent mutliple lines added after repeated provision executions) + grep -qxF 'export KUBECONFIG=/etc/rancher/k3s/k3s.yaml' ~/.bashrc || echo 'export KUBECONFIG=/etc/rancher/k3s/k3s.yaml' >> ~/.bashrc + ##must wait web interface of k3s is available to prevent errors on provisioning + wait $PID_K3S_INSTALLATION +fi +echo "Pods running:" +k3s kubectl get pods --all-namespaces -o wide + #cleanup in case of halted vm wich is reprovisioning k3s kubectl delete deployments --all k3s kubectl delete services --all diff --git a/vagrant/pyvoltdpsimdemo.sh b/vagrant/pyvoltdpsimdemo.sh index 51a0028..0b652c7 100644 --- a/vagrant/pyvoltdpsimdemo.sh +++ b/vagrant/pyvoltdpsimdemo.sh @@ -1,5 +1,23 @@ #!/usr/bin/env bash +#installing k3s +if ! command -v k3s &> /dev/null +then + echo "------- K3S could not be found let's install it -------" + curl -sfL https://get.k3s.io | sh - + sudo chown -R vagrant /etc/rancher/k3s/ + sudo helm --kubeconfig /etc/rancher/k3s/k3s.yaml list + ##getting pid of installation to wait until completed + PID_K3S_INSTALLATION=$! + export KUBECONFIG=/etc/rancher/k3s/k3s.yaml + #appending variable to bashrc if not exists (to prevent mutliple lines added after repeated provision executions) + grep -qxF 'export KUBECONFIG=/etc/rancher/k3s/k3s.yaml' ~/.bashrc || echo 'export KUBECONFIG=/etc/rancher/k3s/k3s.yaml' >> ~/.bashrc + ##must wait web interface of k3s is available to prevent errors on provisioning + wait $PID_K3S_INSTALLATION +fi +echo "Pods running:" +k3s kubectl get pods --all-namespaces -o wide + #sogno-demo DATABUS: install rabbitmq via helm # The `rabbitmq_values.yaml` file contains SOGNO specific overwrites of the default rabbitMQ values. cat > /etc/rancher/k3s/rabbitmq_values.yaml<< EOF diff --git a/vagrant/simulationdemo.sh b/vagrant/simulationdemo.sh index a071b30..612b8e9 100644 --- a/vagrant/simulationdemo.sh +++ b/vagrant/simulationdemo.sh @@ -1,92 +1,123 @@ -#!/usr/bin/env bash - -cd example-deployments/simulation-demo -set -o nounset -set -o errexit - -#SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) -#pushd $SCRIPT_DIR && echo "Changed to $SCRIPT_DIR" - -#cleanup in case of halted vm wich is reprovisioning -k3s kubectl delete deployments --all -k3s kubectl delete services --all -k3s kubectl delete pods --all --grace-period=0 --force -k3s kubectl delete daemonset --all - -echo "Starting rabbitmq" -k3s kubectl apply -f ./rabbitmq/deployment.yaml --kubeconfig /etc/rancher/k3s/k3s.yaml -k3s kubectl apply -f ./rabbitmq/service.yaml --kubeconfig /etc/rancher/k3s/k3s.yaml - -REDISISRUNNING=`k3s kubectl --kubeconfig /etc/rancher/k3s/k3s.yaml get pod redis-master-0 --output="jsonpath={.status.containerStatuses[*].ready}" | cut -d' ' -f2` -echo "Redis already running? $REDISISRUNNING"; -if [ ! "$REDISISRUNNING" = true ] ; -then - echo "installing redis" - sudo helm install redis --set auth.enabled=false bitnami/redis --kubeconfig /etc/rancher/k3s/k3s.yaml - echo "installed redis" -fi - -echo "Starting minio" -k3s kubectl apply -f ./minio/deployment.yaml --kubeconfig /etc/rancher/k3s/k3s.yaml -k3s kubectl apply -f ./minio/service.yaml --kubeconfig /etc/rancher/k3s/k3s.yaml -k3s kubectl apply -f ./minio/configmap.yaml --kubeconfig /etc/rancher/k3s/k3s.yaml - -#installing minikube -#curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube_latest_amd64.deb -#sudo dpkg -i minikube_latest_amd64.deb -#minikube ssh docker pull amazon/aws-cli - -echo "Creating sogno-platform bucket" -k3s kubectl run --kubeconfig /etc/rancher/k3s/k3s.yaml --rm -i --tty aws-cli --overrides=' -{ - "apiVersion": "v1", - "kind": "Pod", - "metadata": { - "name": "aws-cli" - }, - "spec": { - "containers": [ - { - "name": "aws-cli", - "command": [ "/root/.aws/setup" ], - "spacer": [ "bash" ], - "image": "amazon/aws-cli", - "stdin": true, - "stdinOnce": true, - "tty": true, - "volumeMounts": [ - { - "mountPath": "/root/.aws", - "name": "credentials-volume" - } - ] - } - ], - "volumes": [ - { - "name": "credentials-volume", - "configMap": - { - "name": "aws-config", - "path": "/root/.aws", - "defaultMode": 511 - } - } - ] - } -} -' --image=amazon/aws-cli --restart=Never -- - - -echo "Starting file service" && -k3s kubectl apply -f ./file-service/deployment.yaml -k3s kubectl apply -f ./file-service/configmap.yaml - - -echo "Starting dpsim api" && -sudo helm install dpsim-api sogno/dpsim-api --kubeconfig /etc/rancher/k3s/k3s.yaml -echo "Starting dpsim worker" && -sudo helm install dpsim-worker sogno/dpsim-worker --kubeconfig /etc/rancher/k3s/k3s.yaml - -echo "Pods running:" -k3s kubectl get pods --all-namespaces -o wide \ No newline at end of file +#!/usr/bin/env bash +cd /home/vagrant +#first of all we need to install minikube +curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 +sudo install minikube-linux-amd64 /usr/local/bin/minikube + +getent group docker ||sudo groupadd docker +sudo usermod -aG docker $USER +sudo usermod -aG docker vagrant && newgrp docker #bash scritps are run as root on provisioning + +#sudo chown "$USER":"$USER" /home/"$USER"/.docker -R +#sudo chmod g+rwx "$HOME/.docker" -R + + +#minikube requires golang and cri-dockerd +cd /home/vagrant +git clone https://github.com/Mirantis/cri-dockerd.git +sudo apt-get install golang -y +cd /home/vagrant/cri-dockerd +mkdir -p bin +go get && go build -o bin/cri-dockerd +mkdir -p /usr/local/bin +sudo install -o $USER -g $USER -m 0755 bin/cri-dockerd /usr/local/bin/cri-dockerd +sudo cp -a packaging/systemd/* /etc/systemd/system +sudo sed -i -e 's,/usr/bin/cri-dockerd,/usr/local/bin/cri-dockerd,' /etc/systemd/system/cri-docker.service +sudo systemctl daemon-reload +sudo systemctl enable cri-docker.service +sudo systemctl enable --now cri-docker.socket + +#helm requires the kubectl configuration stored in ~/.kube/config so it must be available also for root +mkdir -p /root/.kube +cp /home/vagrant/.kube/config /root/.kube/config +chmod 644 /root/.kube + +minikube kubectl -- get po -A #install compliant version of kubectl +minikube start --force + +cd /home/vagrant/example-deployments/simulation-demo +set -o nounset +set -o errexit + +#SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) +#pushd $SCRIPT_DIR && echo "Changed to $SCRIPT_DIR" + +#cleanup in case of halted vm wich is reprovisioning +#kubectl delete deployments --all +#kubectl delete services --all +#kubectl delete pods --all --grace-period=0 --force +#kubectl delete daemonset --all + +#echo "Starting rabbitmq" +kubectl apply -f /home/vagrant/example-deployments/simulation-demo/rabbitmq/deployment.yaml +kubectl apply -f /home/vagrant/example-deployments/simulation-demo/rabbitmq/service.yaml + +REDISISRUNNING=`kubectl get pod redis-master-0 --output="jsonpath={.status.containerStatuses[*].ready}" | cut -d' ' -f2` +echo "Redis already running? $REDISISRUNNING"; +if [ ! "$REDISISRUNNING" = true ] ; +then + echo "installing redis" + sudo helm install redis --set auth.enabled=false bitnami/redis + echo "installed redis" +fi + +#echo "Starting minio" +kubectl apply -f /home/vagrant/example-deployments/simulation-demo/minio/deployment.yaml +kubectl apply -f /home/vagrant/example-deployments/simulation-demo/minio/service.yaml +kubectl apply -f /home/vagrant/example-deployments/simulation-demo/minio/configmap.yaml +# + +minikube ssh docker pull amazon/aws-cli +# +echo "Creating sogno-platform bucket" +kubectl run --rm -i --tty aws-cli --overrides=' +{ + "apiVersion": "v1", + "kind": "Pod", + "metadata": { + "name": "aws-cli" + }, + "spec": { + "containers": [ + { + "name": "aws-cli", + "command": [ "/root/.aws/setup" ], + "spacer": [ "bash" ], + "image": "amazon/aws-cli", + "stdin": true, + "stdinOnce": true, + "tty": true, + "volumeMounts": [ + { + "mountPath": "/root/.aws", + "name": "credentials-volume" + } + ] + } + ], + "volumes": [ + { + "name": "credentials-volume", + "configMap": + { + "name": "aws-config", + "path": "/root/.aws", + "defaultMode": 511 + } + } + ] + } +} +' --image=amazon/aws-cli --restart=Never -- + +echo "Starting file service" && +kubectl apply -f /home/vagrant/example-deployments/simulation-demo/file-service/deployment.yaml +kubectl apply -f /home/vagrant/example-deployments/simulation-demo/file-service/configmap.yaml + +echo "Starting dpsim api" && +sudo helm install dpsim-api sogno/dpsim-api +echo "Starting dpsim worker" && +sudo helm install dpsim-worker sogno/dpsim-worker + +echo "Pods running:" +minikube kubectl get po -A \ No newline at end of file diff --git a/vagrant/vagrant-basic-bootstrap.sh b/vagrant/vagrant-basic-bootstrap.sh index 0326aa8..ea0709b 100644 --- a/vagrant/vagrant-basic-bootstrap.sh +++ b/vagrant/vagrant-basic-bootstrap.sh @@ -1,79 +1,57 @@ -#!/usr/bin/env bash - -# Increase No of HPgs as required from sogno demo documentation -echo 1024 | sudo tee /proc/sys/vm/nr_hugepages - -sudo ufw disable -apt-get update -apt-get install -y apt-transport-https ca-certificates curl gnupg2 software-properties-common net-tools - - -#installing docker -if ! command -v docker &> /dev/null -then - echo "------- Docker could not be found let's install it -------" - curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - - sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" - sudo apt-get install -y docker-ce - # adding user vagrant to docker group - sudo usermod -aG docker $USER - sudo systemctl enable docker - sudo systemctl start docker -fi - -#installing kubernetes -if ! command -v kubelet &> /dev/null -then - echo "------- Kubernetes could not be found let's install it -------" - curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add - - sudo bash -c 'echo "deb http://apt.kubernetes.io/ kubernetes-xenial main" > /etc/apt/sources.list.d/kubernetes.list' - sudo apt-get update - sudo sudo apt-get install -y kubelet kubeadm kubectl - sudo systemctl enable kubelet - sudo systemctl start kubelet -fi - -##installing helm -if ! command -v helm &> /dev/null -then - echo "------- Helm could not be found let's install it -------" - curl https://baltocdn.com/helm/signing.asc | gpg --dearmor | sudo tee /usr/share/keyrings/helm.gpg > /dev/null - echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/helm.gpg] https://baltocdn.com/helm/stable/debian/ all main" | sudo tee /etc/apt/sources.list.d/helm-stable-debian.list - sudo apt-get update - sudo apt-get install -y helm - sudo helm repo add bitnami https://charts.bitnami.com/bitnami - helm repo add sogno https://sogno-platform.github.io/helm-charts - helm repo add influxdata https://influxdata.github.io/helm-charts - helm repo add grafana https://grafana.github.io/helm-charts - helm repo update -fi -# - -##installing k3s -if ! command -v k3s &> /dev/null -then - echo "------- K3S could not be found let's install it -------" - curl -sfL https://get.k3s.io | sh - - sudo chown -R vagrant /etc/rancher/k3s/ - sudo helm --kubeconfig /etc/rancher/k3s/k3s.yaml list - ##getting pid of installation to wait until completed - PID_K3S_INSTALLATION=$! - export KUBECONFIG=/etc/rancher/k3s/k3s.yaml - #appending variable to bashrc if not exists (to prevent mutliple lines added after repeated provision executions) - grep -qxF 'export KUBECONFIG=/etc/rancher/k3s/k3s.yaml' ~/.bashrc || echo 'export KUBECONFIG=/etc/rancher/k3s/k3s.yaml' >> ~/.bashrc - ##must wait web interface of k3s is available to prevent errors on provisioning - wait $PID_K3S_INSTALLATION -fi - -echo "Pods running:" -k3s kubectl get pods --all-namespaces -o wide - -echo "cluster info" -k3s kubectl cluster-info - -if [ ! -d "example-deployments" ] -then - #checkout examples demo - git clone https://github.com/sogno-platform/example-deployments.git - rm -rf ./example-deployments/vagrant -fi +#!/usr/bin/env bash + +# Increase No of HPgs as required from sogno demo documentation +echo 1024 | sudo tee /proc/sys/vm/nr_hugepages + +sudo ufw disable +apt-get update +apt-get install -y apt-transport-https ca-certificates curl gnupg2 software-properties-common net-tools + + +#installing docker +if ! command -v docker &> /dev/null +then + echo "------- Docker could not be found let's install it -------" + curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - + sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" + sudo apt-get install -y docker-ce + # adding user vagrant to docker group + sudo usermod -aG docker $USER + sudo systemctl enable docker + sudo systemctl start docker +fi + +#installing kubernetes +if ! command -v kubelet &> /dev/null +then + echo "------- Kubernetes could not be found let's install it -------" + curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add - + sudo bash -c 'echo "deb http://apt.kubernetes.io/ kubernetes-xenial main" > /etc/apt/sources.list.d/kubernetes.list' + sudo apt-get update + sudo apt-get install -y kubelet kubeadm kubectl + sudo systemctl enable kubelet + sudo systemctl start kubelet +fi + +##installing helm +if ! command -v helm &> /dev/null +then + echo "------- Helm could not be found let's install it -------" + curl https://baltocdn.com/helm/signing.asc | gpg --dearmor | sudo tee /usr/share/keyrings/helm.gpg > /dev/null + echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/helm.gpg] https://baltocdn.com/helm/stable/debian/ all main" | sudo tee /etc/apt/sources.list.d/helm-stable-debian.list + sudo apt-get update + sudo apt-get install -y helm + sudo helm repo add bitnami https://charts.bitnami.com/bitnami + helm repo add sogno https://sogno-platform.github.io/helm-charts + helm repo add influxdata https://influxdata.github.io/helm-charts + helm repo add grafana https://grafana.github.io/helm-charts + helm repo update +fi +# + +if [ ! -d "example-deployments" ] +then + #checkout examples demo + git clone https://github.com/sogno-platform/example-deployments.git + rm -rf ./example-deployments/vagrant +fi From bfbbcd84b317e89065fddbbd938498c914359c3d Mon Sep 17 00:00:00 2001 From: Federico Paolantoni <1670962+paolantoni@users.noreply.github.com> Date: Wed, 17 Aug 2022 17:45:01 +0200 Subject: [PATCH 09/12] fix user owner of cri-dockerd riequired by minikube and edited the last command that lists pods. Signed-off-by: Federico Paolantoni --- vagrant/simulationdemo.sh | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/vagrant/simulationdemo.sh b/vagrant/simulationdemo.sh index 612b8e9..0735d82 100644 --- a/vagrant/simulationdemo.sh +++ b/vagrant/simulationdemo.sh @@ -20,7 +20,7 @@ cd /home/vagrant/cri-dockerd mkdir -p bin go get && go build -o bin/cri-dockerd mkdir -p /usr/local/bin -sudo install -o $USER -g $USER -m 0755 bin/cri-dockerd /usr/local/bin/cri-dockerd +sudo install -o vagrant -g vagrant -m 0755 bin/cri-dockerd /usr/local/bin/cri-dockerd sudo cp -a packaging/systemd/* /etc/systemd/system sudo sed -i -e 's,/usr/bin/cri-dockerd,/usr/local/bin/cri-dockerd,' /etc/systemd/system/cri-docker.service sudo systemctl daemon-reload @@ -119,5 +119,10 @@ sudo helm install dpsim-api sogno/dpsim-api echo "Starting dpsim worker" && sudo helm install dpsim-worker sogno/dpsim-worker + +#passing configuration to minikube kubectl commands +mkdir -p /home/vagrant/.kube +sudo kubectl config view --raw >/home/vagrant/.kube/config + echo "Pods running:" -minikube kubectl get po -A \ No newline at end of file +sudo kubectl get pods -o wide \ No newline at end of file From 86e953fd41e45ed53d6e106283ae700f7379a547 Mon Sep 17 00:00:00 2001 From: Federico Paolantoni <1670962+paolantoni@users.noreply.github.com> Date: Wed, 17 Aug 2022 17:46:26 +0200 Subject: [PATCH 10/12] use >> instead > on appending something in existing file Signed-off-by: Federico Paolantoni --- vagrant/simulationdemo.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/vagrant/simulationdemo.sh b/vagrant/simulationdemo.sh index 0735d82..e191205 100644 --- a/vagrant/simulationdemo.sh +++ b/vagrant/simulationdemo.sh @@ -122,7 +122,7 @@ sudo helm install dpsim-worker sogno/dpsim-worker #passing configuration to minikube kubectl commands mkdir -p /home/vagrant/.kube -sudo kubectl config view --raw >/home/vagrant/.kube/config +sudo kubectl config view --raw >> /home/vagrant/.kube/config echo "Pods running:" sudo kubectl get pods -o wide \ No newline at end of file From aa3db9f7bf34e4c77def97eb668a5cc421e5df35 Mon Sep 17 00:00:00 2001 From: Federico Paolantoni <1670962+paolantoni@users.noreply.github.com> Date: Thu, 18 Aug 2022 15:11:23 +0200 Subject: [PATCH 11/12] adding the description about the authors into readme. Signed-off-by: Federico Paolantoni --- README.md | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 9a92ad6..8ce4078 100644 --- a/README.md +++ b/README.md @@ -5,10 +5,14 @@ More detailed descriptions and high-level architecture descriptions are availabl ## Running examples using vagrant and virtualbox -We currently provide the following examples. -More examples to come soon! +This fork introduces vagrant to help setting up a ready to use environment with examples and has been developed at + + AReti areti + to further ease the execution. + +Each example runs in a vm (8gb ram, 2cpu). +We currently provide the following examples: - [PMU Data Visualization](pmu-data-visualization) execute: `cd vagrant && vagrant up pmudatavisualization` - [Pyvolt DPsim Demo](pyvolt-dpsim-demo) execute: `cd vagrant && vagrant up pyvoltdpsim` - [Simulation Demo](simulation-demo) execute: `cd vagrant && vagrant up simulation` - From 6c7be4eea4be66497a484a7e357fe8a4add08168 Mon Sep 17 00:00:00 2001 From: Federico Paolantoni <1670962+paolantoni@users.noreply.github.com> Date: Thu, 18 Aug 2022 17:01:56 +0200 Subject: [PATCH 12/12] included hint for vmware provider Signed-off-by: Federico Paolantoni --- README.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/README.md b/README.md index 8ce4078..d9c82cf 100644 --- a/README.md +++ b/README.md @@ -16,3 +16,7 @@ We currently provide the following examples: - [PMU Data Visualization](pmu-data-visualization) execute: `cd vagrant && vagrant up pmudatavisualization` - [Pyvolt DPsim Demo](pyvolt-dpsim-demo) execute: `cd vagrant && vagrant up pyvoltdpsim` - [Simulation Demo](simulation-demo) execute: `cd vagrant && vagrant up simulation` + +In case VMware is preferred please run appending: + +`cd vagrant && vagrant up --provider vmware_desktop` \ No newline at end of file