SkinSync - AI Skincare Assistant

SkinSync is an AI-powered skincare assistant prototype built to support personalized skin-care guidance through a conversational interface. The implemented chatbot uses retrieval-augmented generation (RAG), biomedical document embeddings, Qdrant similarity search, and a locally hosted quantized medical language model. The broader FYP vision includes image-based skin analysis, product recommendations, diet guidance, progress tracking, and user feedback.

Medical disclaimer: SkinSync is an educational and support-oriented prototype. It does not replace dermatologists, medical diagnosis, professional treatment, or emergency care.

System Architecture

The architecture above shows the full project vision: a user-facing web or mobile experience, login/signup flow, chatbot page, backend services, a fine-tuned medical language model, and an image classification system for skin-condition analysis.

Current Prototype

This repository currently contains the chatbot/RAG slice of SkinSync:

• FastAPI web app serving a simple browser-based chat interface.
• LangChain RetrievalQA pipeline for grounding answers in indexed documents.
• Qdrant vector database for similarity search over skincare and biomedical PDF content.
• PubMedBERT embeddings via NeuML/pubmedbert-base-embeddings.
• Local BioMistral GGUF model served through llama_cpp / LangChain.
• Source-aware responses that return an answer, retrieved context, and source document metadata.

The prototype flow is:

1. Add medical/skincare reference documents locally.
2. Run ingest.py to split, embed, and store document chunks in Qdrant.
3. Start the FastAPI app from app.py.
4. Ask skincare-related questions through the web UI.
5. Receive grounded answers with retrieved context.

Chatbot Core

The current implementation focuses on a local, privacy-conscious chatbot pipeline rather than a hosted API model:

Layer	Implementation
User input	Browser form posts a skincare query to /get_response
Document loading	DirectoryLoader scans local PDF files from data/
Chunking	RecursiveCharacterTextSplitter with chunk_size=700 and chunk_overlap=70
Embeddings	NeuML/pubmedbert-base-embeddings, a biomedical embedding model based on PubMedBERT
Vector storage	Qdrant collection vector_db_2
Vector config	768-dimensional vectors with cosine similarity
Retrieval	LangChain retriever with k=1, returning the most relevant document chunk
Prompting	Context-aware prompt that tells the model to avoid fabricated answers
Generation	RetrievalQA chain with chain_type="stuff"
Response payload	Answer, retrieved context, and source document path

Model And Token Configuration

The chatbot uses BioMistral-7B.Q4_K_M.gguf as the local language model. The Q4_K_M suffix means the model is stored in a 4-bit GGUF quantized format, using the K-quants medium preset. This keeps the model small enough for local CPU-friendly experimentation while preserving better answer quality than very aggressive quantization options.

Setting	Value	Purpose
Model file	BioMistral-7B.Q4_K_M.gguf	Local BioMistral 7B model
Quantization	Q4_K_M	Balanced 4-bit quantization for size, speed, and quality
Runtime	llama_cpp_python through LangChain LlamaCpp	Runs the GGUF model locally
Temperature	0.3	Keeps responses focused and less random
max_tokens	2048	Caps generated answer length
top_p	1	Uses full nucleus sampling range
Retriever depth	k=1	Grounds each answer in the top matching chunk

The prompt format injects retrieved context before the user question, then asks the model to return only a helpful, detailed answer. If the retrieved knowledge is insufficient, the prompt instructs the model to say that it does not know instead of inventing an answer.

Full FYP Vision

The final SkinSync concept is broader than the current codebase. Based on the project report, the complete assistant is designed around four modules:

• Disease prediction and image processing: preprocess uploaded skin images, classify likely skin conditions, and provide confidence scores.
• User interface: support authentication, chat, image upload, progress history, and recommendation views across web and mobile.
• Interactive chatbot: answer skin-care questions, use contextual information, and collaborate with diagnostic and recommendation modules.
• Product and diet recommendations: suggest skincare products, supplements, and dietary changes based on skin type, detected conditions, history, and user needs.

Prototype vs Proposed Design

Area	Current repository	Proposed/FYP design
Frontend	HTML/Jinja template served by FastAPI	React-based login, signup, landing, and chatbot pages
Backend	FastAPI endpoint at /get_response	Flask/backend service layer shown in the design
LLM	Local BioMistral Q4_K_M GGUF model via llama.cpp	Fine-tuned LLaMA-style medical model / Ollama-style LLM engine
Retrieval	Qdrant + PubMedBERT embeddings	Medical dataset processing and fine-tuning pipeline
Image analysis	Documented in report, not implemented in this repo	Feature extraction and image classifier module
Recommendations	Chatbot can answer from retrieved sources	Dedicated product, diet, and supplement recommendation engine

Tech Stack

• Python
• FastAPI
• Jinja2
• LangChain
• Qdrant
• Sentence Transformers
• PubMedBERT biomedical embeddings (NeuML/pubmedbert-base-embeddings)
• BioMistral 7B local GGUF model (Q4_K_M quantized)
• Llama.cpp Python bindings

Project Structure

SkinSync/
+-- app.py                         # FastAPI chatbot application
+-- ingest.py                      # PDF ingestion and Qdrant indexing
+-- requirements.txt               # Python dependencies
+-- templates/
|   +-- index.html                 # Chat UI
+-- assets/
|   +-- architecture-design.png    # System architecture diagram
+-- README.md                      # Portfolio documentation

Local-only files such as model weights, vector database state, virtual environments, and private/reference PDFs are intentionally excluded from GitHub.

Local Reproduction Notes

This repository is presented as a portfolio prototype. To recreate the chatbot locally, provide your own permitted medical/skincare PDF documents, download a compatible local GGUF model, run Qdrant locally, then run the ingestion and FastAPI app.

Example commands:

pip install -r requirements.txt
python ingest.py
uvicorn app:app --reload

The application expects:

• A running Qdrant instance at http://localhost:6333.
• A local model file named BioMistral-7B.Q4_K_M.gguf or an updated model_path in app.py.
• User-provided source PDFs in a local data/ folder.

Limitations

• This is not a clinical diagnostic system.
• The current repository implements the chatbot/RAG prototype, not the complete image classifier or product recommendation engine.
• Chronic or severe skin conditions are outside the intended prototype scope.
• Accuracy depends on the quality, safety, and relevance of the indexed source documents.
• Privacy, security, model evaluation, and scalability would need further work before real-world use.

Future Work

• Add a trained image classifier for acne, pigmentation, dryness, and other skin-condition categories.
• Build a structured recommendation engine for products, diet, vitamins, and minerals.
• Add user accounts, chat history, feedback loops, and progress tracking.
• Improve response evaluation with precision, recall, safety checks, and dermatology-reviewed test cases.
• Package the system with a cleaner local setup and deployment workflow.

License

No public license is included yet. Ownership and reuse permissions should be decided before publishing this repository for external reuse.