File permission issues cause backups to stop being created

[silverdr]

I encountered a situation, where backups never finish, leaving the in_progress directory always there. After analysing, the culprit seemed to be a specific file, which had 0000 permissions set. Changing the permissions allowed backups to work normally. Such situations should probably be handled more gracefully. While zero permissions look fishy, I can imagine other permissions related situations that would also cause failures. Like files created when running processes with elevated privileges (sudo <something>).

[silverdr]

Another situation:

rsync: send_files failed to open "<path_to_file>/04B81CFC0A0102ABs": Permission denied (13)
rsync: send_files failed to open "<path_to_file>/19228148A9E5D347s": Permission denied (13)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1207) [sender=3.1.3]

and backups stop being created.

[cytopia]

which had 0000 permissions set

With this permission it is expected that a user (non-root) is unable to read the file.

handled more gracefully

What should be a suitable behaviour for the backup solution, while still letting the user know, that something has not been backed up?

[silverdr]

What should be a suitable behaviour for the backup solution, while still letting the user know, that something has not been backed up?

That's obviously a good question. "Thinking aloud" - first that comes to my mind is to mark the directory with "_errors" at the end of its name, and placing a text file named the same as the failed backup dir plus "_errors.txt" next to the directory. To keep it simple, the file might contain just the stderr output like the one I posted above (I assume it's stderr output). This way it would be immediately clear that not everything went 100% correct but the errors wouldn't prevent backups from finishing, potentially leading to data losses.

[jschwalbe]

Came here to report the same problem. Had a directory that was causing the following error:

rsync: opendir "/home/ob1/docker-configs/tandoor/postgresql" failed: Permission denied (13)
IO error encountered -- skipping file deletion
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1207) [sender=3.1.3]
2022-12-29 22:57:15 timemachine: [ERROR] timemachine Backup has failed

I had to chown it to my user (instead of root). Nevertheless, this error stopped a whole bunch of data/files from being copied, and had been doing so for about 20 days. Yikes!

1. I would agree that this could be handled more gracefully. (At least continue to copy the other files.)
2. I clearly need some checks and balances to see if the process is running to completion; do you have any suggestions?

Thanks!

[jschwalbe]

Whatever docker container I've got doing this, seems to keep making root owned files. Don't like it, but I'm resorting to running linux-timemachine as root otherwise it never finishes.

[cytopia]

I also always run it as root. I will dig in on how to be able to continue when rsync has errors.

[m4p1e]

I suggest that tool should give warnings to these fails and continue working. Because that is really annoyed, the backup will never finish...

[TylerRick]

I had a file owned by root with 000 permissions, too, on a shared hosting server I was backing up...

---------- 1 root        root        280 Mar 22  2022 /.../.htaccess.cpmh3129

Even though it aborted early with this error:

rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1865) [generator=3.2.7]
2023-09-27 12:21:53 timemachine: [ERROR] timemachine Backup has failed

, it appears that it continued copying files into the in_progress dir.

While I was a bit disappointed that it aborted the entire huge backup and said it "failed", over 1 insignificant file which I wouldn't have cared if it was skipped, I can see how this script can't know whether a file failing to copy meets the user's idea of a "successful" backup or not. It depends entirely on which file(s) it is and how important that file is to the user.

I wouldn't be opposed to a "--allow-failed-files" option or similar that allows them to override the default behavior and consider it a success even if one or more files failed. Hopefully the user is monitoring the stderr output anyway, to watch for any warnings or errors. That way they would both be aware of the problem (to fix it for later backups) and have the backup considered successful and permanently saved in the regularly scheduled timestamped backup dir.

But I think it would be better if users would use excludes/filters to simply skip any problematic files so they don't cause any error in the first place... (With #81, however, it sounds like it may not always be possible to know which files will be problematic ahead of time.)

---

Since it is a shared hosting account where I don't have root, I couldn't simply change the permissions as the OP did; I needed a way to exclude those files from the transer, similar to #64.

I ended up passing --filter='dir-merge /.rsync-filter' to the rsync command (see MR #86 for proposal to auto-add that, to make it easier for people to exclude problem files ) so that I could add a .rsync-filter file with - .htaccess.cpmh3129 in the source tree.

[book]

When running my own rsync script for backups, I regularly see this error, usually caused by temporary files that disappeared before the backup was complete.

rsync warning: some files vanished before they could be transferred (code 24) at main.c(1333) [sender=3.2.3]

rsync has a comprehensive list of exit statuses that correspond to the different errors encountered during transfer.

It might be useful to consider all the possible exit codes, and decide which ones are not sufficient to declare a backup failed. The full list is provided in the rsync manual page:

EXIT VALUES
    0      Success
    1      Syntax or usage error
    2      Protocol incompatibility
    3      Errors selecting input/output files, dirs
    4      Requested action not supported: an attempt was made to manipulate 64-bit
           files on a platform that cannot support them; or an option was specified
           that is supported by the client and not by the server.
    5      Error starting client-server protocol
    6      Daemon unable to append to log-file
    10     Error in socket I/O
    11     Error in file I/O
    12     Error in rsync protocol data stream
    13     Errors with program diagnostics
    14     Error in IPC code
    20     Received SIGUSR1 or SIGINT
    21     Some error returned by waitpid()
    22     Error allocating core memory buffers
    23     Partial transfer due to error
    24     Partial transfer due to vanished source files
    25     The --max-delete limit stopped deletions
    30     Timeout in data send/receive
    35     Timeout waiting for daemon connection

23 and 24 definitely look like they are not sufficient to declare the backup worthless.