Re: What is the recommended backup strategy for GlusterFS?

[Aravinda <avishwan@xxxxxxxxxx>]

We have one more tool. glusterfind!

This tool comes with gluster installaton, if you are using Gluster 3.7.  glusterfind enables Changelogging(Journal) to Gluster Volume and uses that information to detect the changes happened in the Volume.

1. Create a glusterfind session using, glusterfind create <SESSION_NAME> <VOLUME_NAME>
2. Do a full backup.
3. Run glusterfind pre command to generate the output file with the list of changes happened in Gluster Volume after glusterfind create. For usage information glusterfind pre --help
4. Consume that output file and backup only the files listed in output file.
5. After consuming the output file, run glusterfind post command. (glusterfind post --help)

Doc link: http://gluster.readthedocs.org/en/latest/GlusterFS%20Tools/glusterfind/index.html

This tool is newly released with Gluster release 3.7, please report issues or request for features here https://bugzilla.redhat.com/enter_bug.cgi?product=GlusterFS

regards
Aravinda

On 09/06/2015 12:37 AM, Mathieu Chateau wrote:

Hello,

for my needs, it's about having a simple "photo" of files present 5 days ago for example.

But i do not want to store file data twice, as most file didn't change.

Using snapshot is convenient of course, but it's risky as you loose both data and snapshot in case of failure (snapshot only contains delta blocks).

Rsync with hardlink is more resistant (inode stay until last reference is removed)

But interested to hear about production setup relying on it

Cordialement,
Mathieu CHATEAU
http://www.lotp.fr

2015-09-05 21:03 GMT+02:00 M S Vishwanath Bhat <msvbhat@xxxxxxxxx>:

MS
On 5 Sep 2015 12:57 am, "Mathieu Chateau" <mathieu.chateau@xxxxxxx> wrote:
>
> Hello,
>
> so far I use rsnapshot. This script do rsync with rotation, and most important same files are stored only once through hard link (inode). I save space, but still rsync need to parse all folders to know for new files.
>
> I am also interested in solution 1), but need to be stored on distinct drives/servers. We can't afford to loose data and snapshot in case of human error or disaster.
>
>
>
> Cordialement,
> Mathieu CHATEAU
> http://www.lotp.fr
>
> 2015-09-03 13:05 GMT+02:00 Merlin Morgenstern <merlin.morgenstern@xxxxxxxxx>:
>>
>> I have about 1M files in a GlusterFS with rep 2 on 3 nodes runnnig gluster 3.7.3.
>>
>> What would be a recommended automated backup strategy for this setup?
>>
>> I already considered the following:

Have you considered glusterfs geo-rep? It's actually for disaster recovery. But might suit your backup use case as well.

My two cents

//MS

>>
>> 1) glusterfs snapshots in combination with dd. This unfortunatelly was not possible so far as I could not find any info on how to make a image file out of the snapshots and how to automate the snapshot procedure.
>>
>> 2) rsync the mounted file share to a second directory and do a tar on the entire directory after rsync completed
>>
>> 3) combination of 1 and 2. Doing a snapshot that gets mounted automaticaly and then rsync from there. Problem: How to automate snapshots and how to know the mount path
>>
>> Currently I am only able to do the second option, but the fist option seems to be the most atractive.
>>
>> Thank you for any help on this.
>>
>> _______________________________________________
>> Gluster-users mailing list
>> Gluster-users@xxxxxxxxxxx
>> http://www.gluster.org/mailman/listinfo/gluster-users
>
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users@xxxxxxxxxxx
> http://www.gluster.org/mailman/listinfo/gluster-users

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users