On Sat, Jan 2, 2021 at 6:27 AM Patrick O'Callaghan <pocallaghan@xxxxxxxxx> wrote: > > Brief recap: I'm currently using BorgBackup, which among other things > does compression and deduplication. I'm looking at the pros and cons of > changing to Btrfs for backup (I already use it as my main fs). The > backup drives are currently ext4 so would need to be converted. > > Now read on. > > Following Chris Murphy's suggestion, I looked at: > > https://fedoramagazine.org/btrfs-snapshots-backup-incremental/ > > My question is whether I can use compression and deduplication on the > backup even if these have not been enabled on the original fs. My > instinct is to say no, but I could be wrong. You can use compression and/or dedup on the backup even if they are not used on the original file system (whether ext4 or Btrfs). Compression is just a mount option on Btrfs. [1] The current version of the Btrfs send stream is not compressed. [2] Btrfs dedup is "out-of-band" in that the file system is mounted, but the dedup is not live. Writes, including duplicates, get written to disk, and are later submitted for deduplication by a utility like duperemove (in Fedora repo) or Bees. [3] The way reflinks (efficient copies), snapshots, and dedup relate to each other, is they all result in shared extents. The actions for arriving at shared extents differ quite a bit. What makes btrfs different from other file systems, and how it relates to send/receive, is it has the ability to cheaply compute differences between snapshots, without requiring deep traversal on either the send or receive file systems. The less your /home changes, the more convenient it is to keep the backup up to date, because the cost of figuring out what to back up is so chea. If your ~/ contains many things that change that you don't care about backing up, then most any other backup solution provides "excludes" to filter out such things. Btrfs send/receive doesn't have such a thing. The closest approximation is you can create a nested subvolume in place of a directory to create a snapshot boundary, because btrfs snapshots are not recursive into subvolumes. e.g. you could create a new ~/.cache subvolume to substitute the directory. It could be a ~1G of savings just for the Firefox cache alone. Note: subvolumes have their own pool of inodes, they're separate file trees not separate file systems, but can sometimes behave like a separate file system. Programs like rsync and find will include subvolumes by default unless you use -x or -xdev respectively. There's an equivalent option in borgbackup which is enabled by default. So it excludes nested subvolumes and snapshots by default. Part of the reasoning for this is that folks with many snapshots were finding that their backup program sees each snapshot as a completely separate and massive pile of files subject to being backed up. Sooo yeah... something to watch out for. If you ask borg to backup /, therefore it will not automatically backup /home because it's considered a separate "file system". Whew... -- Chris Murphy [1] You can change the compression option while mounted by using the remount option along with the new compression values you want. Mix and match is valid. Whatever is set at the time of writing, is what's being used. If you forget to mount with compression, then those writes aren't compressed. Etc. [2] There are patches upstream that will persist compression in the stream; it's preliminary work to support native encryption which will likewise preserve encryption in the send stream. This means we'll be able to replicate snapshots even if they aren't unlocked, and the replication uses the same key as the original. Stay turned. [3] https://github.com/Zygo/bees -- Chris Murphy _______________________________________________ users mailing list -- users@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to users-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@xxxxxxxxxxxxxxxxxxxxxxx
