Script and tips for parallelizing rsync

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi,

I recently had a RAID failure on one of my Gluster replicas; luckily my
replica was ok, and I could re-sync all the data to the bad node's
bricks.  I used rsync to pre-seed the brick data, rather than having
Gluster's self-heal daemon try to figure it out.

It turns out I had way more files than I realized, which exposed some
problems with "traditional" rsync invocation.  I found some clever ways
to optimize the transfer and speed up the process, and wrote up my
experiences on my blog:

http://mjanja.co.ke/2014/07/parallelizing-rsync/

Hope this helps someone!

-- 
Alan Orth
alan.orth@xxxxxxxxx
http://alaninkenya.org
http://mjanja.co.ke
"I have always wished for my computer to be as easy to use as my telephone; my wish has come true because I can no longer figure out how to use my telephone." -Bjarne Stroustrup, inventor of C++
GPG public key ID: 0x8cb0d0acb5cd81ec209c6cdfbd1a0e09c2f836c0


Attachment: signature.asc
Description: OpenPGP digital signature

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://supercolony.gluster.org/mailman/listinfo/gluster-users

[Index of Archives]     [Gluster Development]     [Linux Filesytems Development]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Bugtraq]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux