Harpocrates

This work is part of my 📝 final year dissertation at the University of Glasgow

This is the mono-repository for the Harpocrates sensitive document redaction Proof of Concept application. Harpocrates is a Web Application for redacting sensitive documents with Machine Learning (ML) classification of documents by sensitivity as part of my final year project. It is a ReactJS frontend backed by an OpenAPI generated Flask API with a Scikit-learn ML model to classify text documents stored in MongoDB. I am using Lime to extract features explaining the ML classification and to present a classification aid with its explanation to redactors in a friendly redaction web app.

Project Overview

How to run

To run the Harpocrates stack, you will need:

• docker
• docker-compose

git clone https://gitlab.com/harpocrates-app/harpocrates.git
cd harpocrates
docker-compose up -d

Prebuilt docker images

A Docker image repository with prebuilt docker images is also available here

Documentation

Documentation for Harpocrates services is available here: https://harpocrates-app.gitlab.io/harpocrates

Development

The development Logbook of Harpocrates can be found here.

Tech stack

Develop locally (without docker)

To run the stack without docker you will need to have:

• Python 3.7.x
• NodeJS 13.x
• MongoDB 4.2.x

Clone the repository

git clone https://gitlab.com/harpocrates-app/harpocrates.git
cd harpocrates
cat hosts.txt >> /etc/hosts

Create and install backend API environment and run API

cd api
pip install --user pipenv
pipenv install
pipenv shell
flask run --port 80

Create and install frontend Node environment and run development build of frontend

cd frontend
npm install
npm run dev

Run a MongoDB instance

you can use the one define in the docker-compose :

docker-compose up -d mongo

Generate the Javascript API Client code

The js-api-client service is a Node package auto-generated using the openapi-generator from the specification defined in api-sepecification/api-specification/openapi.yml.

To update the JS client after having modified the specification use the following command at the repo's root after having installed the OpenAPI code generator:

make api-client

Generate the Flask API Server code

The Flask API server code is also derived from the OpenAPI generator code, however, some files are protected from "auto generation overwrite" this list is in api/.openapi-generator-ignore. As such, some manual modification of these files might be required after code generation which is run from the project root with the following command after having installed the OpenAPI code generator:

make api-server