Skip to content

feat: filesystem directory backup#27

Draft
a-mesin wants to merge 9 commits into
mainfrom
dir-backup
Draft

feat: filesystem directory backup#27
a-mesin wants to merge 9 commits into
mainfrom
dir-backup

Conversation

@a-mesin

@a-mesin a-mesin commented Nov 12, 2023

Copy link
Copy Markdown
Member

closes #16

Source directory of the backup is getting archived in a tar archive and compressed with gzip resulting in a .tar.gzip file at the configured destination.

Still missing:

  • check if tests are needed and if yes implement them

@a-mesin a-mesin changed the title Filesystem directory backup feat: filesystem directory backup Nov 12, 2023
@a-mesin

a-mesin commented Nov 12, 2023

Copy link
Copy Markdown
Member Author

Caveat: filename in destination config needs to have the file type .tar.gz. This will not be needed once #26 is implemented

@iabudiab

Copy link
Copy Markdown
Member

Hey there, two questions / points:

  • Why not filepath.WalkDir?
  • What about symlinks? They are not followed, but the callback is still called for symlinks, if I'm not mistaken. For example, rsync -a preserve symlinks among other things. lifeboat should offer this option too.

And on the topic of rsync, maybe it can be used as an external tool if available. You know, reinventing the wheel and yada yada 😉

@a-mesin

a-mesin commented Nov 12, 2023

Copy link
Copy Markdown
Member Author

Hey there, two questions / points:

  • Why not filepath.WalkDir?
  • What about symlinks? They are not followed, but the callback is still called for symlinks, if I'm not mistaken. For example, rsync -a preserve symlinks among other things. lifeboat should offer this option too.

And on the topic of rsync, maybe it can be used as an external tool if available. You know, reinventing the wheel and yada yada 😉

Hey,

first of all thanks for having a look :)

  • looks like filepath.WalkDir is indeed the better choice here as appearently it is a more performant/efficient version of filepath.Walk - thanks for pointing it out
  • Symlinks are a complicated one in our use case as we want to support abitrary destinations for a backup. What happens f.e. when the file should be "backupped" into an S3 where the Symlink might point to a non-existing file? One idea could be to actually include the followed file into the final tar?

As for using rsync: the idea was to depend on as few external tools as possible so it can be used in as many environments as possible without the need for other tools.

@a-mesin

a-mesin commented Nov 12, 2023

Copy link
Copy Markdown
Member Author

Just checked the symlink issue: looks like resolving the symlink is quite trivial - therefore I would just add them...

@iabudiab

Copy link
Copy Markdown
Member

Regarding the symlinks, I would do something similar to the tar utility itself, i.e. include the symlinks as soft-links in the archive. Here we have two cases:

  • either the links are relative and point to some file inside the archive => they would work after unpacking
  • or they are absolute and point to something outside => they would be broken after unpacking

With tar -h such links can be dereferenced

@a-mesin a-mesin added the enhancement New feature or request label Nov 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement backup of an directory

2 participants