Chewett Site Sync

Small backup utility for pulling:

• MySQL databases via mysqldump
• Filesystems via rsync
• InfluxDB backups via the Influx CLI
• FTP content via ftplib (FTPBackupManager is still incomplete / experimental)

The entrypoint is sync.py, which reads configuration from backup_details.json in the repository root.

Features

• Daily, weekly, and monthly backup rotation
• Optional table whitelist for database backups
• Optional SSH tunnel support for database backups
• .tgz archive output
• Compression via Python's built-in tarfile, gzip, or pigz

Python Requirements

• Python 3
• Python packages from requirements.txt:
  • mysql-connector-python
  • sshtunnel

Install into a local virtual environment:

python -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install -r requirements.txt

On bash shells:

python3 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.txt

External Tools

Some backup modes also require external executables. These are not installed by pip.

• Database backups: mysqldump
• Rsync backups: rsync and ssh
• InfluxDB backups: influx
• Optional compression modes:
  • compression = "python" uses only Python
  • compression = "gzip" requires tar
  • compression = "pigz" requires tar and pigz

If a tool is not on PATH, point the corresponding config value at its full executable path.

Configuration

1. Copy backup_details_example.json to backup_details.json.
2. Fill in the sections you need.
3. Run the sync script.

Example:

Copy-Item backup_details_example.json backup_details.json
python sync.py

The config file has four top-level sections:

• settings: shared settings such as mysqldump, tmp_folder, compression, and influx_cli
• dbs: MySQL backup definitions
• ftp: FTP backup definitions
• rsync: rsync backup definitions
• influx: InfluxDB backup definitions

Useful config fields:

• store_location: destination root for that backup set
• backup_file_format: currently tgz
• daily_backups, weekly_backups, monthly_backups:
  • 0 disables that cadence
  • positive integer keeps that many archives
  • -1 keeps unlimited archives for that cadence

Output Layout

Each backup target writes into its store_location using subfolders such as:

• day/
• week/
• month/
• tmp/
• current/ for rsync backups