Skip to content
BranchesTags

Repository files navigation

Remembrall

Hack NC State x Code Black — Preserving Black Culture Track

Remembrall is an assistive desktop application designed to support individuals who may struggle with memory or daily independence — inspired by the magical object from Harry Potter that reminds you when you've forgotten something. The app combines face recognition, location tracking, AI-powered activity logging, and emergency contact features into a single heads-up display (HUD) built with PyQt5.

Table of Contents

Features

Feature Description
User Sign-up Registers a user with personal details (name, age, gender, contact, home location, emergency contact, race/ethnicity) and stores them in the backend API.
Face Recognition Uses OpenCV and a Haar Cascade classifier to detect and recognize the user's face via a live camera feed.
Location Tracking Retrieves the user's current GPS coordinates (via IP geolocation), calculates the distance to their registered home address, and renders both points on an interactive Folium map saved as footprint.html.
Location Display Shows a static location map image inside a PyQt5 widget.
AI Activity Log Records 30 seconds of audio, transcribes it using OpenAI Whisper, and generates a keyword + summary with GPT-3.5-turbo. Each log entry is time-stamped and stored in the user's profile via the REST API.
Task Board Fetches the user's activity log from the API and displays it as a scrollable, checkable task list with the current date and time.
SOS / Emergency Panel Provides one-tap buttons to call the user's caretaker, trigger an SOS alert, or return home.
Community Ticker A draggable, semi-transparent text-input widget that keeps a scrollable history log of entered notes.

Architecture

┌─────────────────────────────────────────────────────┐
│                  PyQt5 Desktop App                  │
│                                                     │
│  start.py ──► signup.py      (User Registration)   │
│           └──► testcam.py    (Face Recognition)    │
│                taskdate.py   (Task Board / HUD)    │
│                soscall.py    (SOS / Emergency)     │
│                community.py  (Community Ticker)    │
│                locationdisplay.py (Map Display)    │
└───────────────────────┬─────────────────────────────┘
                        │ HTTP (requests)
                        ▼
        ┌───────────────────────────────┐
        │  Django REST API (Heroku)     │
        │  remembrall-cd7db0135b38      │
        │  .herokuapp.com               │
        └───────────────────────────────┘
                        ▲
                        │
        ┌───────────────┴───────────────┐
        │   log_summarizer/             │
        │   recordAudio → Whisper STT   │
        │   → GPT-3.5 summarize → API  │
        └───────────────────────────────┘

Tech Stack

Frontend / Desktop

  • Python 3
  • PyQt5 — GUI framework
  • OpenCV (cv2) — Camera capture & face detection
  • Folium — Interactive maps rendered as HTML

AI / ML

Location

  • geocoder — IP-based GPS coordinate lookup
  • geopy — Address geocoding & distance calculation

Backend

Audio

Setup & Installation

Prerequisites

1. Clone the repository

git clone https://github.com/akulka404/hack-ncstate.git
cd hack-ncstate

2. Install desktop app dependencies

pip install PyQt5 opencv-python folium geocoder geopy requests pyaudio openai pytz

3. Configure the OpenAI API key

The log summarizer reads the API key from log_summarizer/api.py. Create or update that file:

# log_summarizer/api.py
def api():
    return "YOUR_OPENAI_API_KEY"

4. (Optional) Run the backend locally

cd remembrall
pip install -r requirements.txt
python manage.py migrate
python manage.py runserver

Usage

Launch the main application

python start.py

This opens the Remembrall start screen with Login and Sign-up options.

Sign up a new user

Click Sign-up to open the registration form. Fill in all fields and click Sign-up User. This will:

  1. POST the user data to the backend API.
  2. Kick off face-recognition training (train.py) for the new user.

Run the activity logger

cd log_summarizer
python main.py <username>

This will:

  1. Record 30 seconds of audio from the microphone.
  2. Transcribe the audio with OpenAI Whisper.
  3. Summarize the transcript with GPT-3.5 and append a time-stamped entry to the user's log in the API.

View the task board

python taskdate.py

Displays the user's logged activities as a scrollable, semi-transparent HUD with checkboxes.

Track location

python location.py

Generates footprint.html showing current location vs. home address on an interactive map.

API

The REST API is deployed at:

https://remembrall-cd7db0135b38.herokuapp.com
Endpoint Method Description
/user/ POST Create a new user
/user/<username>/ GET Retrieve a user's data
/user/<username>/update PUT Update a user's log
/admin GET Django admin panel

Project Structure

hack-ncstate/
├── start.py                  # App entry point (Login / Sign-up screen)
├── start.ui                  # Qt Designer UI file for start screen
├── signup.py                 # User registration form
├── signup.ui                 # Qt Designer UI file for sign-up
├── taskdate.py               # Task board / HUD widget
├── taskdate.ui               # Qt Designer UI file for task board
├── soscall.py                # SOS / emergency contact panel
├── soscall.ui                # Qt Designer UI file for SOS panel
├── community.py              # Draggable community text-ticker widget
├── testcam.py                # Live camera feed with face detection
├── location.py               # GPS location tracking & map generation
├── locationdisplay.py        # Static map image display widget
├── locationdisplay.ui        # Qt Designer UI file for location display
├── footprint.html            # Generated Folium map (current ↔ home)
├── location.png              # Static location map image
├── haarcascade_frontalface_default.xml  # OpenCV face detector model
├── log_summarizer/
│   ├── main.py               # Entry point: record → transcribe → log
│   ├── recordAudio.py        # PyAudio microphone recorder (30 s)
│   ├── transcribe.py         # OpenAI Whisper transcription
│   ├── summarize.py          # GPT-3.5 keyword + summary generation
│   ├── logger.py             # Fetches user log, appends entry, PUTs back
│   ├── api.py                # Returns the OpenAI API key
│   ├── transcription.txt     # Latest transcription output
│   └── output.wav            # Latest recorded audio
└── remembrall/               # Django backend
    ├── manage.py
    ├── requirements.txt
    ├── Procfile              # Heroku process file
    ├── runtime.txt           # Python runtime for Heroku
    ├── remembrall/           # Django project settings
    └── user/                 # Django app: User model & REST endpoints

Hackathon Progress

Track: Preserving Black Culture

  • API Development
  • API Deployment (Heroku)
  • Face Recognition
  • Log Generator (Audio → Whisper → GPT-3.5 → API)
  • Heads-up Display (all components complete)
  • Presentation
  • Video

About

Hack_NCState x Code Black

hackncstate.org/

Resources

Readme
MIT license
Activity

Stars

0 stars

Watchers

1 watching

Forks

0 forks
Report repository

Used by

Contributors

Languages