Does this tool take backup in an incremental fashion

[hakimkartik]

Say i have cluster on day 1 having 2 keyspaces , having a combined size of 100G, using the tool i triggered the backup and uploaded 100G of data to s3 by the end of day.
On day 2 say my cluster receives new data and so the total size of the cluster become about 200G(100G of new data).
So when i trigger the backup for day 2
will the tool start uploading data from the beginning (i.e. all of 200G)
or will only upload the new data(100G)

[perfectayush]

@hakimkartik The answer is a bit complicated. Scyllabackup uploads backup of snapshots created using nodetool snapshot command. These snapshots contain sstable files which are immutable.

In your example, you have mentioned adding 100 GB data added on the second day. Assuming you have taken a snapshot the day before, when nodetool snapshot will be run on the second day, new sstable files will be generated, but some of the old sstable files will get removed. Between two snapshots there may be many common sstable files, which will not be reuploaded, if the backup is being taken in same storage prefix. Only the new files will be uploaded by scyllabackup. Storage/creation of sstable files in a snapshot is not under control of scyllabackup, it's optimized by scylladb. So I can't say how many the new sstable files will be created. Scylladb also merges sstable files to create bigger files for various optimization, in that case, more new data will be uploaded.

I want to clarify though that scyllabackup doesn't support incremental backups as supported by scylladb.

To re-iterate, each backup taken by scyllabackup is a new snapshot which results in creation of some new sstable files and removal of some old sstable files. The common files are not reuploaded, if present remotely. The extent of new files depends on scylladb.