[C4GT] Create Telemetry dashboard for notifications sent out

[karntrehan]

Project Details

UCI is created with a vision to make the conversation flow creation configurable to a level, that it can be done by program owners, without the intervention of any engineers at all. At the same time, it also envisions reducing the redundant coding required for enabling the same business logic for different channels.

UCI aims to democratize the use of different communication channels such as WhatsApp, Telegram, SMS, email and more across all governance use cases.

UCI is being used to create 2 ways communication channels via chatbots between lakhs of teachers in Uttar Pradesh & the state. These chatbots are configured from an admin panel and trigger notifications to the teachers. Teachers then click on the notifications on their Android phones and talk to the chatbot. This is making day to day communication realtime & very effective between teachers and the state.

Features to be implemented

Once a notification is received, frontend clients can send back telemetry / analytics to the UCI acknowledging delivery and read receipts of the notifications. Currently this telemetry is not visualised.

We need to plan and implement a dashboard which will visualise the data to the users.

This could be integrated in the UCI - Admin repo itself.

What exists

• firebase/web API already exists in inbound. Declared here. The data is being inserted into a database as well.

  Payload for API:

  {
    "text": "",
    "from": "",
    "messageId": "",
    "eventType": "DELIVERED/READ",
    "report": {
      "externalId": "",
      "destAdd": "",
      "fcmDestAdd": ""
    }
  }

• Apart from this, the notification monitoring script outputs log data and Kafka data in a JSON format given below:

  {
    "Kafka Topics": {
      "broadcast-transformer": "7",
      "com.odk.transformer": "17",
      "generic-transformer": "0",
      "notification-outbound": "35",
      "outbound": "17",
      "inbound-processed": "24",
      "process-outbound": "52",
      "telemetry": "14",
      "outbound-processed": "0"
    },
    "UCI API": {
      "ResolveUser": 1
    },
    "Inbound": {
      "Data inserted in Cassandra": 1,
      "Kafka topic push": 1
    },
    "Orchestrator": {
      "Topic consumed from Kafka": 5,
      "Notification pushed to Kafka": 5
    },
    "Broadcast Transformer": {
      "Broadcast Processed": 5,
      "Kafka Push to outbound": 5
    },
    "Outbound": {
      "Saved data in cache": 5,
      "Notification inserted in Cassandra": 5
    }
  }
  

  The number of sections and items within each section can change. This needs to be visualized on the dashboard where each section should have its own graph illustrating its own items data.

• All the services also generate log files which should be viewable from the dashboard itself. The log files may contains several GBs of data hence it is crucial to handle the pooling of data from the frontend and limit the amount of data being downloaded or else, it may be detrimental to the system. The functionality should consist of the following:

  • Pooling log data from the server.
  • Filtering logs based on a search string.

What needs to be built

1. Reading data from the database
2. Adding visualization to uci - admin
3. Reading data from 1 and showing in 2

Learning Path

Complexity

Medium

Skills Required

Java to understand BE systems, React to design and develop the dashboard.

Name of Mentors:

@chinmoy12c

Project size

8 Weeks

Product Set Up

https://github.com/samagra-comms/docker-deploy#manual-setup---one-click-deployment-is-wip

Acceptance Criteria

• Data sent from frontend clients is shown in the dashboard
• Dashboard is filterable by:
  • Bot Id / Bot name
  • Event type - Delivered / Read
  • Date
  • External Id

Out of Scope

1. Implementation of events in frontend

Milestones

• Understanding the requirements
• Understanding UCI Inbound
• Understanding UCI Admin
• Setting up UCI locally
• Reading data from inbound database
• Planning visualization
• Adding visualization to uci - admin
• Reading data from database and showing in dashboard
• Adding filtering to dashboard
• Adding test cases to dashboard and consumer

Designs

We shall refer to open source posthog visualizations.

C4GT

This issue is nominated for Code for GovTech (C4GT) 2023 edition.
C4GT is India's first annual coding program to create a community that can build and contribute to global Digital Public Goods. If you want to use Open Source GovTech to create impact, then this is the opportunity for you! More about C4GT here: https://codeforgovtech.in/

[Harsh06-afk]

I am interested in this issue and would like to contribute in this as I have experience in Java and React.js

[khushia2512]

I want to contribute to this project. I have experience in Java and React.js.

[kunal00000]

Hi @karntrehan, I would like to work on this project.
Just want to get some things clarified -

1. We want to visualise analytics/data (in the form of charts or graph maybe) in UCI-admin dashboard.

• This data is fetched from inbound API.
• Add various filter options for data.

2. Is the tech stack written above correct as I saw UCI-admin repo is built with Angular and not React?

[DishaJain17]

Hello everyone,My name is Disha Jain, and I'm excited to be a part of this project. I'm currently looking for open issues to contribute to, and I would greatly appreciate it if you could suggest some for me to work on.

I have a strong technical background, particularly in the MERN stack, Java, C++, Linux, Git, and I also have a solid understanding of basics of YAML, Docker, and Kubernetes. I'm eager to utilise my skills and knowledge to contribute meaningfully to this project.I am also a technical blogger and i have written so many tutorial blogs .

Resume:https://drive.google.com/file/d/1-mPdea8ZHQv7B19y0v8CDHl3rEJsowie/view?usp=drivesdk

Intro:https://drive.google.com/file/d/1-8wHZN-0Ew3aojcU3xJm9MNUoWWrAoAI/view?usp=drivesdk

Blogs:https://hashnode.com/@disha390

LinkedIn: https://www.linkedin.com/in/disha-jain-90a554229

Thank you in advance for your support and recommendations. I'm looking forward to hearing from you soon!

[Deepak21573]

Hi @karntrehan I would like to express my strong proficiency in, HTML, CSS, and Node.js , Javascript and React.js. With a solid grasp of these technologies, I am eager to make a meaningful contribution to this project. If possible, I kindly request to be assigned to this endeavor so that I may commence my efforts promptly and begin actively working on it.
Also can i get to know about the original github link the project is currently working on ??