Skip to content

Commit eb6dc2b

Browse files
author
Spencer
committed
updates
1 parent 0463658 commit eb6dc2b

1 file changed

Lines changed: 233 additions & 32 deletions

File tree

‎README.md‎

Lines changed: 233 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -1,61 +1,262 @@
11
# URL Watcher
22

3-
A simple Python tool that monitors websites for content changes and sends SMS notifications via TextBelt.
3+
A Python tool that monitors websites for content changes and sends SMS notifications via ClickSend.
44

55
## Features
66

7-
- Monitor any URL for content changes
8-
- SMS notifications when changes are detected
9-
- Continuous monitoring with randomized intervals
10-
- Secure credential storage via .env files
11-
- Comprehensive test coverage
7+
- **Single URL Monitoring**: Monitor one URL with simple command-line interface
8+
- **Multi-URL Monitoring**: Monitor multiple URLs with individual check intervals
9+
- **SMS Notifications**: Get notified via ClickSend when content changes
10+
- **Change Detection**: SHA256-based content hashing with diff output
11+
- **Resilience**: Automatic retry logic with exponential backoff (multi-URL mode)
12+
- **Continuous Monitoring**: Background monitoring with randomized intervals
13+
- **Comprehensive Logging**: Detailed logs of all checks and notifications
1214

1315
## Quick Start
1416

15-
1. **Install dependencies:**
16-
```bash
17-
pip install -r requirements.txt
18-
```
17+
### 1. Install Dependencies
1918

20-
2. **Set up SMS notifications (optional):**
21-
Create a `.env` file:
22-
```
23-
SMS_PHONE_NUMBER=+1234567890
24-
TEXTBELT_API_KEY=your_textbelt_api_key
25-
```
19+
```bash
20+
pip install -r requirements.txt
21+
pip install clicksend-client # For SMS notifications
22+
```
2623

27-
3. **Monitor a URL:**
28-
```bash
29-
# Single check
30-
python url_watcher.py https://example.com
24+
### 2. Configure SMS (Optional)
3125

32-
# Continuous monitoring
33-
python url_watcher.py https://example.com --continuous
26+
Create a `.env` file in the project root:
3427

35-
# With SMS alerts
36-
python url_watcher.py https://example.com --sms
37-
```
28+
```bash
29+
SMS_PHONE_NUMBER="+19786341135"
30+
CLICKSEND_USERNAME="your_email@example.com"
31+
CLICKSEND_API_KEY="your-api-key-here"
32+
```
33+
34+
Get your ClickSend credentials at: https://dashboard.clicksend.com/
35+
36+
### 3. Monitor a URL
37+
38+
**Single URL - One-time check:**
39+
```bash
40+
python src/url_watcher.py https://example.com --sms
41+
```
42+
43+
**Single URL - Continuous monitoring:**
44+
```bash
45+
python src/url_watcher.py https://example.com --continuous --sms
46+
```
47+
48+
**Multi-URL monitoring with JSON config:**
49+
```bash
50+
python src/multi_url_watcher.py urls.json --sms
51+
```
52+
53+
**Multi-URL - Single URL with custom interval:**
54+
```bash
55+
python src/multi_url_watcher.py https://example.com --interval 60 --sms
56+
```
3857

3958
## Usage
4059

60+
### Single URL Watcher
61+
4162
```bash
42-
python url_watcher.py <URL> [--continuous] [--sms]
63+
python src/url_watcher.py <URL> [--continuous] [--sms]
4364
```
4465

45-
- `--continuous`: Monitor continuously with random intervals
46-
- `--sms`: Send SMS notifications when changes are detected
66+
**Options:**
67+
- `--continuous` - Monitor continuously with randomized intervals (300-600 seconds)
68+
- `--sms` - Send SMS notifications when changes detected
4769

48-
## Requirements
70+
**Examples:**
71+
```bash
72+
# Check once
73+
python src/url_watcher.py https://example.com
4974

50-
- Python 3.9+
51-
- TextBelt API account (for SMS notifications)
75+
# Continuous monitoring with SMS
76+
python src/url_watcher.py https://example.com --continuous --sms
77+
78+
# Run in background with logging
79+
nohup python src/url_watcher.py https://example.com --continuous --sms 2>&1 | tee -a url_watcher.log &
80+
```
81+
82+
### Multi-URL Watcher
83+
84+
```bash
85+
python src/multi_url_watcher.py <config.json|URL> [--interval SECONDS] [--sms]
86+
```
87+
88+
**Options:**
89+
- `--interval SECONDS` - Check interval (only when using single URL, default: 300)
90+
- `--sms` - Send SMS notifications
91+
92+
**Examples:**
93+
```bash
94+
# Monitor multiple URLs from config file
95+
python src/multi_url_watcher.py urls.json --sms
96+
97+
# Monitor single URL with 60-second interval
98+
python src/multi_url_watcher.py https://example.com --interval 60 --sms
99+
100+
# Run in background
101+
nohup python src/multi_url_watcher.py urls.json --sms 2>&1 | tee -a multi_watcher.log &
102+
```
103+
104+
## Configuration
105+
106+
### Multi-URL JSON Config
107+
108+
Create a `urls.json` file with your URLs and check intervals:
109+
110+
```json
111+
[
112+
{
113+
"url": "https://example.com",
114+
"interval": 300
115+
},
116+
{
117+
"url": "https://api.github.com/repos/python/cpython/releases/latest",
118+
"interval": 3600
119+
},
120+
{
121+
"url": "https://news.ycombinator.com",
122+
"interval": 600
123+
}
124+
]
125+
```
126+
127+
**Configuration options:**
128+
- `url` (required) - The URL to monitor
129+
- `interval` (required) - Check interval in seconds
130+
131+
**Common intervals:**
132+
- `60` - Every minute
133+
- `300` - Every 5 minutes
134+
- `600` - Every 10 minutes
135+
- `1800` - Every 30 minutes
136+
- `3600` - Every hour
137+
- `86400` - Every 24 hours
138+
139+
### Environment Variables
140+
141+
Set these in `.env` file or export them:
142+
143+
```bash
144+
# Required for SMS notifications
145+
SMS_PHONE_NUMBER="+19786341135" # Your phone number with country code
146+
CLICKSEND_USERNAME="user@example.com" # ClickSend username
147+
CLICKSEND_API_KEY="YOUR-API-KEY" # ClickSend API key
148+
149+
# Optional
150+
CLICKSEND_SOURCE="URLWatcher" # Sender name (default: URLWatcher)
151+
```
152+
153+
### Background Monitoring
154+
155+
Run continuously in the background:
156+
157+
```bash
158+
# With screen and file logging
159+
nohup python src/url_watcher.py https://example.com --continuous --sms 2>&1 | tee -a url_watcher.log &
160+
161+
# View live logs
162+
tail -f url_watcher.log
163+
164+
# Check if running
165+
ps aux | grep url_watcher
166+
167+
# Stop the process
168+
pkill -f url_watcher.py
169+
```
170+
171+
**Redirect explanation:**
172+
- `2>&1` - Redirects stderr to stdout (captures all output)
173+
- `| tee -a file.log` - Writes to both screen and log file (appends)
174+
- `&` - Runs in background
52175

53176
## Testing
54177

178+
### Run All Tests
179+
180+
```bash
181+
python -m pytest src/test/ -v
182+
```
183+
184+
### Test SMS Integration
185+
186+
**Verify setup (no SMS sent):**
187+
```bash
188+
pytest src/test/test_sms_integration.py::TestSMSIntegration::test_verify_setup -v
189+
```
190+
191+
**Send actual test SMS:**
55192
```bash
56-
pytest
193+
pytest src/test/test_sms_integration.py::TestSMSIntegration::test_send_test_sms -v
57194
```
58195

196+
**Interactive mode (prompts before sending):**
197+
```bash
198+
python src/test/test_sms_integration.py --interactive
199+
```
200+
201+
See `src/test/SMS_TESTING_GUIDE.md` for detailed testing instructions.
202+
203+
## Features by Watcher Type
204+
205+
### Single URL Watcher (`url_watcher.py`)
206+
- Simple command-line interface
207+
- One URL at a time
208+
- Continuous or one-time checks
209+
- Randomized intervals (300-600s) when continuous
210+
- SMS notifications via ClickSend
211+
212+
### Multi-URL Watcher (`multi_url_watcher.py`)
213+
All features from Single URL Watcher, plus:
214+
- Monitor multiple URLs simultaneously
215+
- Individual check intervals per URL
216+
- Automatic retry with exponential backoff (5s, 10s, 20s)
217+
- Distinguishes network failures from content changes
218+
- SMS alerts when sites go down and recover
219+
- Parallel checking when multiple URLs are due
220+
- Per-URL statistics (check count, last change time)
221+
- Recovery notifications
222+
223+
## Troubleshooting
224+
225+
### SMS not received
226+
227+
1. **Check ClickSend dashboard**: https://dashboard.clicksend.com/sms/history
228+
2. **Verify credentials** in `.env` file
229+
3. **Check phone number format**: Must include country code (e.g., `+19786341135`)
230+
4. **Run verification test**:
231+
```bash
232+
pytest src/test/test_sms_integration.py::TestSMSIntegration::test_verify_setup -v
233+
```
234+
235+
### No changes detected
236+
237+
- First check establishes baseline (no notification sent)
238+
- Subsequent checks compare against cached content
239+
- Cache stored in `url_cache.json` (single) or `multi_url_cache.json` (multi)
240+
241+
### Import errors
242+
243+
Run from project root directory:
244+
```bash
245+
python src/url_watcher.py https://example.com
246+
```
247+
248+
Not from src directory:
249+
```bash
250+
cd src
251+
python url_watcher.py https://example.com # May cause import errors
252+
```
253+
254+
## Requirements
255+
256+
- Python 3.9+
257+
- ClickSend account (for SMS notifications)
258+
- Internet connection
259+
59260
## License
60261

61262
MIT

0 commit comments

Comments
 (0)