This is a web scraper made with Python that runs as cron job with GitHub Actions on a daily basis.
The purpose of this automation is to regularly check for price changes of certain electronic products on selected websites. I'm using the ScrapingDog API to manage the HTTP Requests.
The workflow is scheduled to start running at 00:00 and re-runs every 5 hours. The program's main loop checks the website every hour.
If any price decrease is detected, the program triggers a Telegram bot that responds by sending a message to a private chat.
>>> 2023-08-10 03:59:02 - __main__ - WARNING - Price element not found for site: Pichau
>>> 2023-08-10 04:59:01 - __main__ - INFO - Running the routine. Loop count: 6
>>> 2023-08-10 04:59:02 - __main__ - DEBUG - Requisition status code: 429
>>> 2023-08-10 04:59:02 - __main__ - WARNING - Price element not found for site: Kabum
>>> 2023-08-10 04:59:02 - __main__ - DEBUG - Requisition status code: 429
>>> 2023-08-10 04:59:02 - __main__ - WARNING - Price element not found for site: Pichau
You can check the full log in status.log.
The actions.yml file can be found inside .github/workflows/actions.yml, it was taken from this project.
Check the requirements.txt file for pip third party packages to install.
Note: This project uses secret environment variables that only collaborators can see and use, but you can fork or clone the repo to make your own.
This program is for personal use only, which falls under fair use. This isn't, in any way, used for massive data share, large scale web scraping, nor commercial purposes. It runs in a fair rate limit in order to respect the website's policy.