qna-web

POC application to implement Q&A on a website utilizing the Langchain framework.

Installation

1. Clone the repository.
2. Install python 3.12 virtual environment and activate it.
3. Install Poetry:

   pip install poetry
   

4. Install dependencies

   poetry install
   

5. Create a .env file in the root directory using the provided .env.example file as a template.

Running the application

Crawler

poetry run crawler --max-seen-urls=3 https://gradio.app/

If successful, the crawler will save the URLs and output a message to the console:

INFO - Written 3 urls to gradio.app-urls.txt

You can get usage information by running:

poetry run crawler --help 

Process URLs (data preparation)

⚠️ Ensure that .env file is present in the root directory with the OPENAI_API_KEY variable set.

poetry run process_urls gradio.app-urls.txt

If successful, the process_urls script will output the following messages to the console:

...
INFO - Saved index to html_vector_index
...
INFO - Saved index to text_vector_index
...
INFO - Report saved to report.csv

You can get usage information by running:

poetry run process_urls --help 

Start gradio app

poetry run app

App will be available at http://localhost:7860