Add push-based full-site backup command to Stack2 Connector (zero-disk streaming)

[Copilot]

Stack2 had no way to initiate or stream a WordPress site backup. This adds a backup command to the existing signed command endpoint that streams every file in the WordPress installation directly to the Stack2 API in 10 MB chunks — no ZIP archive is ever written to disk, no local disk accumulation, and no polling required.

New capabilities

• action: backup — initiates a full-site streaming backup (types: database, files, full). Database is dumped in 1,000-row batches to a small temp file; all other files are streamed directly from disk. Pushes each 10 MB chunk to Stack2 immediately as it is read, then deletes the SQL temp file. Returns backup_id, total_chunks, total_bytes, and backup_type.
• action: backup_cleanup — safety net to remove any orphaned SQL temp file if a push fails mid-way.
• GET /wp-json/stack2/v1/backup/{id}/status — returns the outcome of the most recent backup (status: pushed or failed).

Architecture

File	Role
class-stack2-backup-service.php	Zero-disk streaming backup — reads files directly in chunks, hold-back push pattern, SQL temp file management, orphan cleanup
class-stack2-http-client.php	push_backup_chunk() — signs and POSTs each chunk to Stack2
class-stack2-backup-rest-controller.php	HMAC-verified GET status route
BACKUP_API.md	Full Stack2-side integration guide: receiving endpoint contract, signing, file-tree reassembly, restore procedure, and pseudo-code

Backup file tree

Files are streamed directly from WordPress — no ZIP is ever created on disk.

Virtual path	Description	Included in
database/database.sql	Full SQL dump of all WordPress tables	database, full
wordpress/	Entire WordPress root directory — wp-admin/, wp-includes/, wp-content/, wp-config.php, .htaccess, index.php, and any other files present in the WordPress root	files, full

Push flow

WordPress reads each file directly and pushes every 10 MB piece to POST {stack2_base_url}/api/websites/backup/chunk immediately. Each chunk carries file_path and file_chunk_index so Stack2 can reconstruct the original directory tree without reassembling a single archive. The final chunk carries summary metadata (total_chunks, total_bytes, backup_type) so Stack2 knows when to finalise. Signing uses:

POST:stack2-backup-chunk:{timestamp}:{sha256_of_body}

A hold-back pattern ensures the last chunk is always identified correctly: the plugin buffers one chunk ahead and only pushes it when the next read confirms more data exists, so is_last=true can be attached without a separate pass.

Supporting changes

• The only temp file is the SQL dump (wp-content/uploads/stack2-backups/{backup_id}.sql.tmp), which is deleted immediately after its chunks are pushed. Orphaned SQL temp files are cleaned up by an hourly WP-Cron hook (stack2_backup_cleanup_event). The directory is protected by .htaccess.
• Admin panel (Settings → Stack2 Connector) gains a Last Backup section showing colour-coded status (pushed / failed), timestamp, backup ID, total bytes streamed, and any error.
• Seven wp_options keys track backup state: stack2_last_backup_at, _status, _id, _size, _error, _type, _total_chunks.
• No PHP extension beyond the standard library is required (ZipArchive is not needed).

[leighharro]

@copilot this could create a massive amount of chunks which in turn could exceed file limits.
Please simplify this by using larger chunk sizes and let the plugin push the chunks to the api as they are created unless you think the api should definitely pull the chunks.
It should be simple to reassemble the chunks on the other end too.
200 rows of sql is probably a bit small, increase this too.

[Copilot]

@copilot this could create a massive amount of chunks which in turn could exceed file limits.
Please simplify this by using larger chunk sizes and let the plugin push the chunks to the api as they ar...

Done in commit 87653a6. Here's what changed:

• Chunk size increased from 1 MB to 10 MB per chunk (constant CHUNK_SIZE = 10485760).
• Push-based: the plugin now pushes each chunk directly to POST {stack2_base_url}/api/websites/backup/chunk as it reads the file — Stack2 no longer needs to poll or fetch chunks. The temp file is deleted from WordPress immediately after the last chunk is sent.
• SQL batches increased from 200 to 1,000 rows per query (constant SQL_BATCH_SIZE).
• The final chunk carries summary metadata (total_chunks, file_size, file_checksum, backup_type) so Stack2 knows when to finalise reassembly.
• The old chunk-pull GET endpoint (/backup/{id}/chunk/{n}) has been removed. The status endpoint (/backup/{id}/status) is kept for optional verification.
• BACKUP_API.md has been rewritten to document the push flow and the Stack2 receiving endpoint contract (POST /api/websites/backup/chunk), including the signing message and reassembly logic.

[leighharro]

@copilot this backup system only does a backup on uploads.
It should backup the entire Wordpress site so it can be restored in a working state.

[Copilot]

@copilot this backup system only does a backup on uploads.
It should backup the entire Wordpress site so it can be restored in a working state.

Fixed in commit 305b2ad. The backup now creates a standard ZIP archive of the entire WordPress site:

• database/database.sql — full SQL dump of all database tables
• wp-content/ — all files recursively (themes, plugins, uploads, mu-plugins, etc.)
• wp-config.php — WordPress configuration

The backup type files (previously files_manifest) now includes actual file contents, not just a file listing. The full type combines the database dump and all files into a single archive that can be extracted and restored directly using any standard unzip tool.

The temp ZIP is deleted from the WordPress server immediately after being pushed, so no disk space is retained. Requires the PHP zip extension (standard on all WordPress-compatible hosts).

[leighharro]

@copilot this is getting better but the entire Wordpress folder needs to be backed up as there are times where people add other files to that directory

[Copilot]

@copilot this is getting better but the entire Wordpress folder needs to be backed up as there are times where people add other files to that directory

Done in commit 874a997. The backup now archives the entire WordPress root (ABSPATH) under wordpress/ in the ZIP:

backup.zip
├── database/
│   └── database.sql          # Full SQL dump
└── wordpress/
    ├── wp-admin/             # WordPress admin files
    ├── wp-includes/          # WordPress core library files
    ├── wp-content/           # Themes, plugins, uploads, etc.
    ├── wp-config.php
    ├── index.php
    ├── .htaccess
    └── …                     # Any other files placed in the WordPress root

The backup directory itself (wp-content/uploads/stack2-backups/) is excluded to prevent recursive self-inclusion.

[leighharro]

@copilot this is now backing up the correct files and database. Chunking is good. The issue now is that you could have a 10GB site and only have 12gb of storage. So you can't create a backup zip of that you'll likely go over storage quota.
The files need to be chunked and sent but they shouldn't be placed into a large zip first to avoid taking up disk space.

[Copilot]

@copilot this is now backing up the correct files and database. Chunking is good. The issue now is that you could have a 10GB site and only have 12gb of storage. So you can't create a backup zip of th...

Done in commit 95418ec (with a follow-up fix in ee25855). The backup no longer creates any archive on disk at all.

New approach — zero-disk streaming:

• Each file under the WordPress root is opened and read directly in 10 MB pieces, which are pushed to Stack2 immediately.
• The only temp file created is the SQL dump for the database (typically much smaller than the full site). It is deleted right after its chunks are pushed.
• No ZIP is ever written. A 10 GB site with 12 GB of storage now works fine.

Each chunk now carries file_path and file_chunk_index so Stack2 can reconstruct the original directory tree:

chunk payload:
{
  "backup_id":        "bkp_...",
  "chunk_index":      42,          // global sequential index
  "file_path":        "wordpress/wp-content/themes/twentytwenty/style.css",
  "file_chunk_index": 0,           // offset within this file
  "data":             "base64...",
  "checksum":         "sha256...",
  "is_last":          false
}

The final chunk carries total_chunks, total_bytes, and backup_type so Stack2 knows when to finalise.